|
Crinkler by Loonies [web] & TBC
[nfo]
|
||||||||
|---|---|---|---|---|---|---|---|---|
|
|
|||||||
|
popularity : 82% |
|||||||
alltime top: #95 |
|
|||||||
added on the 2005-07-21 14:31:55 by Blueberry  |
||||||||
popularity helper
comments
!
rulez added on the 2005-07-21 15:19:16 by beton 
An executable packer is not a demotool...
I don't know, its a tool created by sceners explicitly for the purpose of efficiently packing 4kb intros. It deserves to be here a lot more than some other "demotool" prods.
Personally, I'm more interested in Blueberry's unreleased Amiga packer...
On a totally unrelated note, the best-named packer in history was "Turtle Smasher". Pure genius!
Personally, I'm more interested in Blueberry's unreleased Amiga packer...
On a totally unrelated note, the best-named packer in history was "Turtle Smasher". Pure genius!
how is a tool used in demo and/or intro-making not a demotool, zoom?
nice idea to include its own linker!
just tested it and it improved my size with only 6 bytes compared to 20to4.. but it seems like it's far more stable then 20to4 atleast ;)
so long all the problem with cab dropping
so long all the problem with cab dropping
and oh, i must ask, doesnt the link.exe trick work with VS 6.0 ?
You shouldn't directly compare filesizes with 20to4 on smaller files (in boozies case ~3000b). Crinkler has an 450byte(~200bytes header) overhead to compensate for before it can gain any bytes compared to 20to4. Crinkler being able to match CAB-dropping on your 3kb file actually shows good promise that you could gain at least 100+bytes compared to 20to4 on a final 4kb file ;)
i just want to say that this packer truly rocks! i love you guys!
a very elegant solution. neat VS integration, neat exe only output. and most importantly, great compression rate.
reflecting to the manual: avoiding dropping is elegant, but cabinet.dll is no cheating (just like opengl32.dll is no cheating either). it is even possible to use cabinet.dll and LZX without writing to the disk. this still has unexposed potential.
reflecting to the manual: avoiding dropping is elegant, but cabinet.dll is no cheating (just like opengl32.dll is no cheating either). it is even possible to use cabinet.dll and LZX without writing to the disk. this still has unexposed potential.
Gem: You are right, using standard Windows dlls is not cheating. But running an external program from a BAT file somehow leaves a bad taste in your mouth. And it doesn't work on my system at all.
Dropping is the thing we are mainly rebelling against. And the best way to do that is to make something competitive.
I doubt you would get good results from using cabinet.dll for in-memory decompression. Compression based on context modelling is sufficiently superior to LZX to make up for the extra 200 bytes of decompression code on a 4k.
Dropping is the thing we are mainly rebelling against. And the best way to do that is to make something competitive.
I doubt you would get good results from using cabinet.dll for in-memory decompression. Compression based on context modelling is sufficiently superior to LZX to make up for the extra 200 bytes of decompression code on a 4k.
nice tool m8, gonna try it
Version 0.2 is now online.
This version fixes a bug in the decompressor, discovered by Frenetic, which would decompress garbage in some "rare" cases, such as when compressing Parsec in SLOW mode. :)
There are also various corrections and updates to the manual and some other minor changes.
Enjoy! :-D
This version fixes a bug in the decompressor, discovered by Frenetic, which would decompress garbage in some "rare" cases, such as when compressing Parsec in SLOW mode. :)
There are also various corrections and updates to the manual and some other minor changes.
Enjoy! :-D
my first feedback: crinkler-.exe was slightly bigger (5000 something) then using a .cab-hack (4800 something)..
good work, maybe i should stop working on 20to4 as obviously crinkler is (at least for me)
a much better solution right now :)
a much better solution right now :)
toxie: Interesting... Are there perhaps any big blocks of zero data in your file?
We would be delighted if you could send us a copy of your obj file that compresses better with cab, so that we can have a look at why we fare so badly.
We would be delighted if you could send us a copy of your obj file that compresses better with cab, so that we can have a look at why we fare so badly.
Blueberry: I will as soon as i think i have a version that's worth it.. Also this was just a quick try (getting it to work somehow), so i bet i will squeeze some few more bytes out of the crinkler-.exe somehow..
anyways: wonderful work!
anyways: wonderful work!
this packer rulez!
Looks promising... but XP only?
The next release version will be Windows 2000 compatible. We have designed a new header layout for that which is actually smaller than the old one when the number of imported functions is low, as is typically the case for DirectX-based intros.
It will probably be some time before the next release, though. A couple of weeks at least, I expect.
It will probably be some time before the next release, though. A couple of weeks at least, I expect.
Really good.
saved my life. really good!
![[self-ban]](https://content.pouet.net/avatars/intro.gif)
I like crinkler very much, but what happend to www.crinkler.net? I have 403 - "You don't have permission to access..." Btw. when will be next release?
Permission problem fixed...
No definite time plan for the next release yet. We might spend some time on it next weekend, which may or may not result in a release.
No definite time plan for the next release yet. We might spend some time on it next weekend, which may or may not result in a release.
wow. big ups for all you 4k gurus releasing stuff like this. it allows incompetent sluts like me to make 4ks that don't even totally suck!! thanks guys (muhmac too, ofc)
how do i get the compression stats in VC6? I turned on compiling debuginfo but it didnt seem to work.
the compression stats feature will be available in version 0.3 of crinkler, but isn't implemented in any publicly released version, yet.
Impressive, I already used it in two prods, and the compression rates are awesome. When's the next version out?
Wow... Knocked nearly 150 bytes off the release version of anorgatronikum! Guess I'll use this instead of my custom cabdropper thingie for my next prod.
Oh my litte p**** waves with joy! This was just awsome and the integration with Visual Studio was amazing!!!! It didn't like HeapAlloc but who cares?
actually best packer in the world... for windows xp.
"Using an external compressor somehow seems like cheating, and this whole dropping business is a mess"
That sentence alone deserves some serious thumb ups :)
That sentence alone deserves some serious thumb ups :)
Version 0.3 is now online.
Crinkler is now also the best packer in the world for win2k and vista :)
This version of crinkler also features E8 call transformations, compression statistics and various other tweaks.
Crinkler is now also the best packer in the world for win2k and vista :)
This version of crinkler also features E8 call transformations, compression statistics and various other tweaks.
Great! I've just tested with same params than 0.2 and i win 28 bytes, really great.
congrats.
congrats.
I lose about 20-30 bytes on all my prods
The compression heuristics used in 0.3 are slightly different from those in 0.2. Some intros get smaller, some get bigger, especially those with many imports (since the new header has space for fewer imports).
Try enabling call transformation. It might very well save you those bytes back (depending on how many calls you have).
Try enabling call transformation. It might very well save you those bytes back (depending on how many calls you have).
Actually, all the intros that I've tested (openGL) get quite bigger with crinkler 0.3, so i'll stick with 0.2, even with call transformation.
Hmm, it really shouldn't get that much bigger.
It would be great if you could send us the object files for the intros you have tried compressing. Then we can investigate what causes the blowup and maybe fix it for the next version.
It would be great if you could send us the object files for the intros you have tried compressing. Then we can investigate what causes the blowup and maybe fix it for the next version.
Neat compactor!
An early christmas present for you: Version 0.4 is online!
As part of our ongoing mission to improve the compatibility of Crinkler, we have now managed to make it compatible with 64-bit versions of Windows.
The new version also includes some other minor improvements and fixes.
As part of our ongoing mission to improve the compatibility of Crinkler, we have now managed to make it compatible with 64-bit versions of Windows.
The new version also includes some other minor improvements and fixes.
Blueberry for president of cruncher country :D
Does it work with VS 2005 now btw?
Does it work with VS 2005 now btw?
We haven't done alot of testing of crinkler with vs2005, but it seems the link.exe drop-in trick
doesn't work directly, as vs2005 doesn't search for link.exe in the project folder before the vs /bin folder.
As a quick fix you can explicitly add the folder containing crinkler(renamed to link.exe) to
the top of your executable path in vs2005.
This is far from an elegant solution, but this is the best we have come up with so far.
We are of course open to suggestions on how to make a more smooth integration with visual studio.
Happy crunching :)
doesn't work directly, as vs2005 doesn't search for link.exe in the project folder before the vs /bin folder.
As a quick fix you can explicitly add the folder containing crinkler(renamed to link.exe) to
the top of your executable path in vs2005.
This is far from an elegant solution, but this is the best we have come up with so far.
We are of course open to suggestions on how to make a more smooth integration with visual studio.
Happy crunching :)
awesome. 'nuff sed.
Version 0.4a is online - just a minor bugfix release.
It seems that some library files (glut32.lib) do not take the coff archive format specification too seriously. Well, you just have to play nice and skip the blanks, and everything works out fine. :)
It seems that some library files (glut32.lib) do not take the coff archive format specification too seriously. Well, you just have to play nice and skip the blanks, and everything works out fine. :)
packed files don't work in win98 or in wine emulation (even if set to winxp) :-( i hope wine poeple will be able to make it work...
This baby gets my thumb of approval
Really useful piece of code, when it comes to 4k prods. :)
selmi: Yes, we scrapped Windows 98 from the start, since it is not relevant for demos nowadays. Just like most other software makers, we consider Windows 98 obsolete.
Not working in WinE is a bit worse, although it doesn't come as a surprise to me. Crinkler EXE files are made for the Windows PE loader (in Windows 2000, XP and Vista), rather than the PE file format specification. The WinE PE loader probably contains a lot more sanity checks than the WIndows PE loader, which is why it rejects Crinkler-compressed files.
Not working in WinE is a bit worse, although it doesn't come as a surprise to me. Crinkler EXE files are made for the Windows PE loader (in Windows 2000, XP and Vista), rather than the PE file format specification. The WinE PE loader probably contains a lot more sanity checks than the WIndows PE loader, which is why it rejects Crinkler-compressed files.
Just today I measured a 4k with apack and with crinkler and crinkler saved an incredible 450 bytes (using GCC). Given you require an additional 450 for your unpacker its an amazing achievement. Great work Mentor and Blueberry. When I get a heart - you get it.
Why don't you Free the memory you use for decompressing? If freeing this Memory uses to much space you could make an option for guys who don't need this space.
The memory used for decompressing overlaps the memory for the resulting executable image itself. So if you allocate your memory statically, it will reuse the memory used for decompressing. If this is still too big, you can reduce it using the /HASHSIZE option.
Here's my thumb up!
Better late than never :)
Better late than never :)
It works with VS.net 2005, and you don't even need the link.exe trick.
Just add a crinkler command-line step in Project Properties / Build Events / Post-Build Event, something like this:
Crinkler.exe /CRINKLER /VERBOSE:IMPORTS /OUT:$(OutDir)\$(InputName)_crinkled.exe <libs> <objects>
Just add a crinkler command-line step in Project Properties / Build Events / Post-Build Event, something like this:
Crinkler.exe /CRINKLER /VERBOSE:IMPORTS /OUT:$(OutDir)\$(InputName)_crinkled.exe <libs> <objects>
PS: Of course in this case you have to include the obj and lib files manually.
I'll use it next week. Hopefuly.
Great tool!
Btw, does it work with Windows Vista?
Btw, does it work with Windows Vista?
I just installed Vista RC2 to find out, and unfortunately the answer is no. We will of course investigate this to figure out what goes wrong and fix it for the next version.
In the meantime, don't worry about your intros not running on Vista if you use Crinkler. We will provide you with some means (such as a recompression option) of making intros compressed using non-Vista-compatible versions run on Vista.
In the meantime, don't worry about your intros not running on Vista if you use Crinkler. We will provide you with some means (such as a recompression option) of making intros compressed using non-Vista-compatible versions run on Vista.
OK, the problem turned out to be very easy to fix. Here is manual fix that you can use to make intros compressed using Crinkler 0.3 or 0.4 Vista-compatible:
Using a hex editor, change 93 into 50 at offset 6A and change 53 into 93 at offset 74. For 0.3-compressed intros, you have to additionally change cdm.dll into lz32.dll at offset 9C. Voala!
Now let's all cross our fingers and hope that Microsoft don't change anything drastic between RC2 and the final release. ;)
Using a hex editor, change 93 into 50 at offset 6A and change 53 into 93 at offset 74. For 0.3-compressed intros, you have to additionally change cdm.dll into lz32.dll at offset 9C. Voala!
Now let's all cross our fingers and hope that Microsoft don't change anything drastic between RC2 and the final release. ;)
Good!
Blueberry: a Vista-fixed version of Crinkler perhaps? (0.4b?)
Crinkler 1.0 pre-announcement:
We are currently finishing the last bits of implementation for Crinkler 1.0, which will be released at tUM 2006. Major new features include:
- Windows Vista compatibility
- Greatly improved compression ratio
- Section reordering to optimize compression
- Big speed improvements
and lots of other stuff.
The new version will be available before the 4k graphics and 4k intro deadlines. Expect to win at least 100 bytes compared to 0.4.
Happy Christmas! :-D
We are currently finishing the last bits of implementation for Crinkler 1.0, which will be released at tUM 2006. Major new features include:
- Windows Vista compatibility
- Greatly improved compression ratio
- Section reordering to optimize compression
- Big speed improvements
and lots of other stuff.
The new version will be available before the 4k graphics and 4k intro deadlines. Expect to win at least 100 bytes compared to 0.4.
Happy Christmas! :-D
Crinkler 1.0 is finally here. Get it on the website now!
Enjoy :)
Enjoy :)
Perfectly timed, ment! Cheers. :D
We use it and it's good!
THANKS.
THANKS.
i have to thumb it, anyhow, since the autors mailserver did not have the best of starts to the new year, i'll try it here :)
that is basically the errormessage:
: fatal error: LNK 0: could not find symbol '__imp___imp_??2@YAPAXI@Z'
intro : error PRJ0003 : Error spawning 'link.exe'.
any hints?
/abuctee
that is basically the errormessage:
: fatal error: LNK 0: could not find symbol '__imp___imp_??2@YAPAXI@Z'
intro : error PRJ0003 : Error spawning 'link.exe'.
any hints?
/abuctee
It was bound to happen: A couple of minor bugs were found and fixed after the 1.0 release, so here is version 1.0a... go get it!
: fatal error: LNK 0: could not find symbol '___ImageBase'
Anyone that knows a solution to this?? Compiled in VS 2005 Express.
tried 'EXTERN_C IMAGE_DOS_HEADER __ImageBase;'
Thanks.
Anyone that knows a solution to this?? Compiled in VS 2005 Express.
tried 'EXTERN_C IMAGE_DOS_HEADER __ImageBase;'
Thanks.
DON'T use the C runtime library startup code. really. just don't.
ryg: do I feel like a real jerk right now?? You bet I am! :(
I declared/defined a virtual destructor in separate h/cpp file. Then i got a linker error on using both vs2003 and vs2005:
: fatal error: LNK 0: could not find symbol '??_EB@@UAEPAXI@Z'
If i remove the virtual keyword it works.
If i declare/define the same in the cpp file where main() is defined, the error also disappears.
Whats the problem?
: fatal error: LNK 0: could not find symbol '??_EB@@UAEPAXI@Z'
If i remove the virtual keyword it works.
If i declare/define the same in the cpp file where main() is defined, the error also disappears.
Whats the problem?
pirx: The problem you are experiencing is due to our lack of support for weak externals in the linker. We will look into this and try to implement this feature in an update for Crinkler.
_THE_ packer for 4ks
kickass
mentor: this sounds good.
Why does a cm-based packer like crinkler need the lempel-ziv lib lz32.dll for decompressing?
Ueber!
cj: We were actually joking about when someone would notice this suspicious looking import.
Crinkler uses hashed importing exclusively and as such doesn't need an import table, but the PE-loader in win2k crashes on executables without at least one import. To fix this we had to include a dummy import table importing some arbitrary function. It just so happens that lz32.dll has the perfect combination of a short name and availability on all windows version :). This also explains why this dll isn't referenced in versions of Crinkler prior to the win2k fix.
Also lz compression isn't really of much use for 4k compression today, even if it is free :D
(People gave up on cab dropping for a reason, even though it has essentially no depacking overhead)
Crinkler uses hashed importing exclusively and as such doesn't need an import table, but the PE-loader in win2k crashes on executables without at least one import. To fix this we had to include a dummy import table importing some arbitrary function. It just so happens that lz32.dll has the perfect combination of a short name and availability on all windows version :). This also explains why this dll isn't referenced in versions of Crinkler prior to the win2k fix.
Also lz compression isn't really of much use for 4k compression today, even if it is free :D
(People gave up on cab dropping for a reason, even though it has essentially no depacking overhead)
Oops, didn't mean to self thumb
Self-thumb away, this rules and you know it ;)
Mentor: Thanks for the answer, it makes sense now. I was just irritated about this library import. :)
amazing tool!
Great stuff. Looking forward to weak externals support.
Mentor, Blueberry, I have a small request that you probably will not like to implement, probably it doesn't fit in Crinkler's way of working but...
as you know floating point numbers compress bad, so people is using the trick of truncating the mantisa with 16 bit of zeros (collapse/necrostudios, farbraush, myself, and probably others). I'm not even sure if Crinkler has enought information to know if a specific constant in the .data segment is a float or not, but would it be possible that you automatically do the truncation for me at linking time? In other to avoid truncating every float (sometimes you don't want truncation for precision issues), only those constants starting like "ct16_" (Crinkler Truncate 16 bits) could be truncated...
I'm saying impossible things? Sorry for asking more from you... Second thumb up to compensate for it :)
as you know floating point numbers compress bad, so people is using the trick of truncating the mantisa with 16 bit of zeros (collapse/necrostudios, farbraush, myself, and probably others). I'm not even sure if Crinkler has enought information to know if a specific constant in the .data segment is a float or not, but would it be possible that you automatically do the truncation for me at linking time? In other to avoid truncating every float (sometimes you don't want truncation for precision issues), only those constants starting like "ct16_" (Crinkler Truncate 16 bits) could be truncated...
I'm saying impossible things? Sorry for asking more from you... Second thumb up to compensate for it :)
another option I would be happy with is to do it per .cpp./obj somehow (like the .obj starting with the indicated prefix) or by function (but this I understand means some logic to inspect the binary the code at link time.
Thanks for the suggestion iq. We are working on that feature now, so you will probably see it in the next release (which will be fairly soon).
A tricky part of it is to recognize float constants that are not marked using a special label. It is possible for float constants in the code, since these are put at a special label by the compiler, but other float constants do not appear to be marked in any special way.
But your suggestion of using the label or file name as indicator is good. Using that will be much easier than truncating the values manually.
A tricky part of it is to recognize float constants that are not marked using a special label. It is possible for float constants in the code, since these are put at a special label by the compiler, but other float constants do not appear to be marked in any special way.
But your suggestion of using the label or file name as indicator is good. Using that will be much easier than truncating the values manually.
Hi. I'm currently writing direct assembly by hand. Would it be possible to have tips/information on crinkler (and it's packing algorithm and segment reordering) so I could perhaps better align the code/data. I expect my data segment to be vastly bigger than the code segment, and the data segment could be split quite freely. Website or the Manual.txt would be a good place for this information.
Thanks again for this great tool. :)
Thanks again for this great tool. :)
XMunkki: Yes, as mentioned in the manual, you should split your code and data into as many sections as possible (that is, only group data that is addressed from the same label). This gives Crinkler most flexibility in choosing a good section ordering.
It is also usually a good idea to give uninitialized sections a high alignment, e.g. 256 bytes or more. This makes your pointers to these sections more similar, resulting in better compression.
It is also usually a good idea to give uninitialized sections a high alignment, e.g. 256 bytes or more. This makes your pointers to these sections more similar, resulting in better compression.
Great tool!!!
When i compile my simple code with masm32, and link it with clinkler, when i try to run i got windows error like "*.exe has encountered an error...", what im compiling is simple program that just use ExitProcess function, it's my first attempt to create as small exe as possible and i simply can't get it to work so i need some help from more experienced user's.
demon: The current version of Crinkler has some problems with very small files (this will be fixed in the next version). Try something a bit bigger, and if you still have problems, mail us your program (or just the object files) and we will have a look at it.
it's indeed amazing stuff ... !!! Hammer !!!
dear team... as far as I see crinkler (version 1.0a) is compatible with XP64 and Vista 64 bit, but not Windows Server 2003 64 bit. Is this a known issue? (if so I can help debugging it). May be Server versions do carefully inspect the exe headers? (that's what you pay for when you have a server version?)
iq: no, we are not aware of any problems with server versions of windows, but we haven't tested them either :)
We would appreciate if you would send us the details of the crash/refusal to run/etc. to us by e-mail. Depending on the nature of the problem, we may, or may not, choose to fix it :)
thanks for your bugreport
We would appreciate if you would send us the details of the crash/refusal to run/etc. to us by e-mail. Depending on the nature of the problem, we may, or may not, choose to fix it :)
thanks for your bugreport
done
This tool rocks. For when the float truncation?
now :)
Although as 256B intro coder I have never used Crinkler, I enjoyed a lot of productions created with using it, and I must say rulez here.
Long, long overdue, but finally, here it is: Crinkler 1.1!
Main new feature is the compression report: A detailed report on the compressed intro with colorcoding to indicate the compression ratio for every byte of code or data. On top of that, it provides disassembly of all the code with all crossreferences as links. Try it!
And then there is the float truncation feature. It is somewhat experimental, as the automatic detection of float constants is necessarily based on heuristics. We are very much interested in feedback on this feature. Is it useful as it is? Any suggestions for improvements?
On the compatibility front, the executables are now compatible with data execution prevention. And, as promised, weak externals are supported now.
The rest are minor fixes and tweaks. See the web page and the manual for the full list.
The compressor itself is unchanged in this release. The only case where the file size will change is if you do not use ranged import. A slightly smaller version of the import code will be used in this case.
Main new feature is the compression report: A detailed report on the compressed intro with colorcoding to indicate the compression ratio for every byte of code or data. On top of that, it provides disassembly of all the code with all crossreferences as links. Try it!
And then there is the float truncation feature. It is somewhat experimental, as the automatic detection of float constants is necessarily based on heuristics. We are very much interested in feedback on this feature. Is it useful as it is? Any suggestions for improvements?
On the compatibility front, the executables are now compatible with data execution prevention. And, as promised, weak externals are supported now.
The rest are minor fixes and tweaks. See the web page and the manual for the full list.
The compressor itself is unchanged in this release. The only case where the file size will change is if you do not use ranged import. A slightly smaller version of the import code will be used in this case.
thanks guys - really terrific.
unbelievable!!!
Will there ever be a gcc version?
Although I'm not a coder I must say "this rocks hard."
Ferris: Which gcc are you using (cygwin/mingw/whatever)? The object format ought to be the same, so if it doesn't work, either of us are doing something wrong. :)
Can you send us some information about what it is you are trying to link with Crinkler and the exact steps you perform to compile and link it? Then we will have a look at it.
Can you send us some information about what it is you are trying to link with Crinkler and the exact steps you perform to compile and link it? Then we will have a look at it.
once more, thanks a lot for this little toy !! :)
yep, that html output helps a lot :)
@Blueberry: I've actually got it to work :D
Dev-C++ outputs the .o files (mirrors of VC++'s .obj files) automatically, so I just made all my code one file and copied VC++'s .lib files that I needed and it works just fine now.
I'm acessing the same dll's so there's no need to use gcc's .a files instead of the .lib files crinkler needs.
All in all I got it to work and it was much easier than I thought. Also, CAB dropping got 1.92kb and this got 1.5...so you are a beast, Crinkler!! :D
Dev-C++ outputs the .o files (mirrors of VC++'s .obj files) automatically, so I just made all my code one file and copied VC++'s .lib files that I needed and it works just fine now.
I'm acessing the same dll's so there's no need to use gcc's .a files instead of the .lib files crinkler needs.
All in all I got it to work and it was much easier than I thought. Also, CAB dropping got 1.92kb and this got 1.5...so you are a beast, Crinkler!! :D
thank you very much, guys. this is impressive :D
@Ferris: I'd be happy to know how you got it to work... I've got some problems with it still using GCC.
Any chance for a Solaris compatible version? Would be awesome to VNC to my uni and abuse their CPU farm.
YEAH...THIS FRIKKIN CRINKLER JUST ROXX.!
THANX A LOT.!!
THANX A LOT.!!
Lord Graga: We use several Windows-specific system calls (such as DLL loading) inside Crinkler itself, so it isn't easily ported to other platforms.
However, you could try running it with Wine. As far as I know, Crinkler works fine with Wine, and Wine is available for (x86) Solaris.
However, you could try running it with Wine. As far as I know, Crinkler works fine with Wine, and Wine is available for (x86) Solaris.
masterpiece)
Speaking of Wine: Latest versions of wine (at least 0.9.59, perhaps also older versions) are now crinkler compatible - correctly running crinkled files.
Great tool without any doubts! Always proving how lame I am at theory of information. I still get surprised by the increased packed size, while original size is decreased by 10-30 bytes of data or code (badly influencing surrounding context?). Also I sometimes get worse rates when I have more data_seg-s. More advices on dealing with this masterpiece would be highly appreciated 8)
Fantastic packer!
Thanks !
Great tool! Question though. I was working on building a very minimal exe using VS 2008. I went the route of putting the crinkler.exe file in my $SolutionDir and renaming it. But, when I run it I keep getting the following error:
fatal error: LNK 0: could not find symbol '__imp__LoadLibraryA@4'
I am not linking against any crt and my project builds fine in Release and debug mode. I am *not* including kernel32.lib in my linker dependencides, which solves the problem. Why is this needed and is there a way I can get around it?
Thanks.
fatal error: LNK 0: could not find symbol '__imp__LoadLibraryA@4'
I am not linking against any crt and my project builds fine in Release and debug mode. I am *not* including kernel32.lib in my linker dependencides, which solves the problem. Why is this needed and is there a way I can get around it?
Thanks.
Great tool!
2 caseyd: I suppose crinkler uses LoadLibrary to avoid import table, so you need to link with kernel32.lib. Good luck ;)
2 caseyd: I suppose crinkler uses LoadLibrary to avoid import table, so you need to link with kernel32.lib. Good luck ;)
Ized: Yeah, that makes sense. Thanks.
Never used this myself but looking how much the quality of Windows 4ks have increased past few years I can only give thumbs up!
I am using it now. Great stuff!
What visa sayed
Hi, thumb up and, of course, I have got newbie question too. I'm using CodeBlocks w/MinGW (thus it's not VC++, because I've problems with registering VC++ 2008 Express Edition). I took aulds OGL Framework from in4k.untergrund.net and placed it in main.c.
Compiling and linking looks like this (/range, /transform-calls and other option temporary disabled, since the code currently is small)
mingw32-gcc.exe -Wall -Os -c main.c -o Release\main.obj
crinkler.exe /subsystem:windows /LIBPATH:"C:\Program Files\Microsoft SDKs\Windows\v6.0A\Lib" /out:Release\AuldOGLFramework.exe opengl32.lib gdi32.lib kernel32.lib user32.lib Release\main.obj /COMPMODE:SLOW /HASHSIZE:256 /HASHTRIES:256 /ORDERTRIES:1024 /UNSAFEIMPORT /PROGRESSGUI
This leds to 641 bytes, through aulds talks about results of around 482 bytes. Is there any switches to compiler and linker I'm missing?
and a second q, I've tried to rename main.c, to main.cpp and got the following error from crinkler:
fatal error: LNK 0: could not find symbol 'WinMainCRTStartup'
Looking into main.obj, I saw this
__Z17WinMainCRTStartupv@0
How to make it look like _WinMainCRTStartup@0?
Compiling and linking looks like this (/range, /transform-calls and other option temporary disabled, since the code currently is small)
mingw32-gcc.exe -Wall -Os -c main.c -o Release\main.obj
crinkler.exe /subsystem:windows /LIBPATH:"C:\Program Files\Microsoft SDKs\Windows\v6.0A\Lib" /out:Release\AuldOGLFramework.exe opengl32.lib gdi32.lib kernel32.lib user32.lib Release\main.obj /COMPMODE:SLOW /HASHSIZE:256 /HASHTRIES:256 /ORDERTRIES:1024 /UNSAFEIMPORT /PROGRESSGUI
This leds to 641 bytes, through aulds talks about results of around 482 bytes. Is there any switches to compiler and linker I'm missing?
and a second q, I've tried to rename main.c, to main.cpp and got the following error from crinkler:
fatal error: LNK 0: could not find symbol 'WinMainCRTStartup'
Looking into main.obj, I saw this
__Z17WinMainCRTStartupv@0
How to make it look like _WinMainCRTStartup@0?
declare your function as 'extern "C" __stdcall void WinMainCRTStartup()'.
@SLeo: he got 482 bytes using dropper, apack and ordinal import turned on.
Using MSVC the best I got here was 614 bytes
Using MSVC the best I got here was 614 bytes
O_o--b
I've got the following error:
.\release\main.obj : fatal error: LNK 0: failed to load file 'F:\TRUE4K\_TRUE4K\MSVC\TRUEINTROSYSTEM\RELEASE\MAIN.OBJ'
the main.obj exist and its in that folder,
I'm using MSVCExpress2008 (v9?) is this the problem? or is the big path ?
.\release\main.obj : fatal error: LNK 0: failed to load file 'F:\TRUE4K\_TRUE4K\MSVC\TRUEINTROSYSTEM\RELEASE\MAIN.OBJ'
the main.obj exist and its in that folder,
I'm using MSVCExpress2008 (v9?) is this the problem? or is the big path ?
solved: "Crinkler does not support the whole program optimization feature of
Visual Studio (General/Whole Program Optimization). Do not turn it on.
"
Visual Studio (General/Whole Program Optimization). Do not turn it on.
"
brilliant stuff!!!
KammutierSpule: Hmm, maybe we should give a slightly more helpful error message in this case. ;)
I'm trying get out of msvcr90.dll with this
/REPLACEDLL:[oldDLL=newDLL]
/REPLACEDLL:msvcr90=msvcrt
but I got the sadly error: fatal error: LNK 0: import '_crt_debugger_hook' could not be found in 'msvcrt'
I know thats because new 90.dll from m$ :S how can we use las VC and use the msvcrt.dll at same time? or at least desactivate the "hook" weird thing..?
/REPLACEDLL:[oldDLL=newDLL]
/REPLACEDLL:msvcr90=msvcrt
but I got the sadly error: fatal error: LNK 0: import '_crt_debugger_hook' could not be found in 'msvcrt'
I know thats because new 90.dll from m$ :S how can we use las VC and use the msvcrt.dll at same time? or at least desactivate the "hook" weird thing..?
Hey, this looks brilliant :P but im having a problem, I linked my intro using Crinlker, without problems in the compiling/linking step, but then when I run the final executable, I get this weird windows error:
Runtime error!
Program: myintro.exe
R6034
An application has made an attempt to load the C runtime library incorrectly.
I'm using VC++2005 in a WinXP SP3 box. Any clue? :)
Thanks for the tool, I'm sure this problem on my intro will eventually be fixed and it was the only way to get it under 4k :P
Runtime error!
Program: myintro.exe
R6034
An application has made an attempt to load the C runtime library incorrectly.
I'm using VC++2005 in a WinXP SP3 box. Any clue? :)
Thanks for the tool, I'm sure this problem on my intro will eventually be fixed and it was the only way to get it under 4k :P
Oh and thumbs up :S
CRiNKLOR!
acidbrain: are you sure that you have only options enabled that are supported by crinkler? E.g. I often forget to turn off the "whole program optimization"-feature in the visual studio options.
oh, and disable the standard runtime library.
4k++ since 2005
a pleasure to work with. I truly love the compression report, it makes it easier to adapt a more compression friendly dataset layout.
Huge hugs!
Huge hugs!
the best demo tool i came across so far. i use it all the time. makes life so much easier...
Another year, another Crinkler! Version 1.1a is now available.
This release focuses on bug fixes and usability. The only change affecting compression rate is a bug fix in the interaction between call transformation and section reordering, which will probably give you a few bytes if you use both.
Apart from that, the long-known float truncation crash bug is now fixed, and there are some improvements to the HTML compression report.
On the usability front, Crinkler now try to detect if some feature is used that Crinkler does not support and gives a helpful hint, e.g. "Disable 'Basic Runtime Checks' in the compiler options". More importantly, there is a new section in the manual which describes how to go about using (or not using) the standard C runtime library with Crinkler. This will hopefully alleviate some of the problems people have had with getting Crinkler to work.
See the web page and the manual for more details on the new version.
If you have any comments, especially if you find cases the manual does not cover or which could use a helpful hint from Crinkler, please tell us. Enjoy!
This release focuses on bug fixes and usability. The only change affecting compression rate is a bug fix in the interaction between call transformation and section reordering, which will probably give you a few bytes if you use both.
Apart from that, the long-known float truncation crash bug is now fixed, and there are some improvements to the HTML compression report.
On the usability front, Crinkler now try to detect if some feature is used that Crinkler does not support and gives a helpful hint, e.g. "Disable 'Basic Runtime Checks' in the compiler options". More importantly, there is a new section in the manual which describes how to go about using (or not using) the standard C runtime library with Crinkler. This will hopefully alleviate some of the problems people have had with getting Crinkler to work.
See the web page and the manual for more details on the new version.
If you have any comments, especially if you find cases the manual does not cover or which could use a helpful hint from Crinkler, please tell us. Enjoy!
when i try to use crinkler on obj created by masm here is what i get :
warning: LNK: Entry point hunk has alignment greater than 1, forcing alignment
of 1
error: LNK: Cannot find symbol '__imp__MessageBoxA@16'
the strange thing is i dont have any MessageBox call inside my asm file
C:\masm32\bin\Link.exe /STACK:0x200000,0x200000 /LIBPATH:"C:\masm32\lib" /SUBSYSTEM:WINDOWS framework.obj => works
crinkler framework.obj /SUBSYSTEM:WINDOWS /ENTRY:EntryPoint /LIBPATH:"C:\masm32\lib" /CRINKLER => error message
warning: LNK: Entry point hunk has alignment greater than 1, forcing alignment
of 1
error: LNK: Cannot find symbol '__imp__MessageBoxA@16'
the strange thing is i dont have any MessageBox call inside my asm file
C:\masm32\bin\Link.exe /STACK:0x200000,0x200000 /LIBPATH:"C:\masm32\lib" /SUBSYSTEM:WINDOWS framework.obj => works
crinkler framework.obj /SUBSYSTEM:WINDOWS /ENTRY:EntryPoint /LIBPATH:"C:\masm32\lib" /CRINKLER => error message
Tigrou: By default the Crinkler decompressor will display an error message (using __imp__MessageBox@A16) if the executable imports from a dll file not on the users system. With the widespread use of the d3dx9_xx family of dlls this is no longer an unlikely scenario.
By using the /UNSAFEIMPORT switch you can change this error message to a simple crash and reduce the decompressor overhead slightly. This is perfectly safe, if you are only using standard dlls.
Crinkler, unlike ms link, doesn't link with any default libraries, so you will have to explicitly link those you, and Crinkler, need. Crinkler needs at least kernel32.lib, but will also need user32.lib if the /UNSAFEIMPORT flag is not used.
If you have further questions about Crinklers quirky behavior, let us know :)
By using the /UNSAFEIMPORT switch you can change this error message to a simple crash and reduce the decompressor overhead slightly. This is perfectly safe, if you are only using standard dlls.
Crinkler, unlike ms link, doesn't link with any default libraries, so you will have to explicitly link those you, and Crinkler, need. Crinkler needs at least kernel32.lib, but will also need user32.lib if the /UNSAFEIMPORT flag is not used.
If you have further questions about Crinklers quirky behavior, let us know :)
Quote:
CRiNKLOR!
I was giving a try to the latest crinkler and I got the following error message:
error PRJ0002 : Error result 3 returned from 'link.exe'
I'm on an xp64 machine, at home with my xp32 it works all fine. Any idea?
error PRJ0002 : Error result 3 returned from 'link.exe'
I'm on an xp64 machine, at home with my xp32 it works all fine. Any idea?
i didn't know this beauty was here!
ups
Please, explain how to use this cool stuff with NASM. Give an example of a simple "MessageBox" application and a command line to build it. Thanks...
scenebas: OK, here's a MessageBox example for NASM:
Assuming you have assembled this code into an object file called mbox.obj and that the LIBPATH is set up correctly (this is the case if you are running a Visual Studio Command Prompt - otherwise you will have to manually point it to your .lib files) the command line to link it is simply
This should result in an out.exe executable of 576 bytes.
Add further options at your leisure.
Does that answer your question?
Code:
extern __imp__MessageBoxA@16
global _entry
section main text align=1
_entry:
push byte 0
push title
push body
push byte 0
call [__imp__MessageBoxA@16]
ret
section str1 data align=1
title: db "Crinkler",0
section str2 data align=1
body: db "Read the manual!",0Assuming you have assembled this code into an object file called mbox.obj and that the LIBPATH is set up correctly (this is the case if you are running a Visual Studio Command Prompt - otherwise you will have to manually point it to your .lib files) the command line to link it is simply
Code:
crinkler mbox.obj /ENTRY:entry kernel32.lib user32.libThis should result in an out.exe executable of 576 bytes.
Add further options at your leisure.
Does that answer your question?
I've been using it all the time and have not yet given my thumb. Really Mentor&Blueberry, you rock! Keep up the great work.
IT ROOOOOOOOOOOOX!!!!
Never had to use (yet) but it's obviously good stuff, nice work.
yes excellent tool thank you!
Thankyouverymuch
thanx for bringing so much support and joy to the demoscene.
ca roule ;)
although im no coder nor ever participated in a 4k, i still believe that crinkler helped make 4k's as good as they are today ..
thumbs up rune and aske ..
thumbs up rune and aske ..
Thumbs up for this great tool!
Thank you for this software :)
I'm having trouble executing any 4k compressed by crinkler under Windows 7, I guess you should take a look at that. It might be a problem of the beta, but still, it could be that MS is changing the way they execute programs.
i can confirm that the current version doesn't work with windows 7... musician complained ;)... btw... at BP i said i weren't using gap contexts... i was wrong there... what i didn't use was context with gaps in them... there can be a gap between the context and the current position... and they are, of course, important for compression
:)
gets the job done. thanks
The Windows 7 problem is present on RC1 as well
Thanks for the reports about the Windows 7 problem. I have investigated it using RC1 and identified the cause of the problem: The digging-into-process-structures hack we use to retrieve the address of kernel32.dll apparently does not work on Windows 7. It gives back the address of a new Windows 7 DLL called KernelBase.dll, so when the import code thinks it is calling LoadLibrary, it is calling some bogus function in KernelBase.dll instead.
We need to investigate this some more to determine how to obtain kernel32 in a way that works on both Windows 7 and earlier Windows. Hopefully we can find a method which is the same size as the old one, since this will greatly simplify the task of fixing old intros to work on Windows 7 (not to mention keeping the size down when adding Windows 7 compatibility).
If all goes well, we will have a fix out before Windows 7 goes public. ;)
We need to investigate this some more to determine how to obtain kernel32 in a way that works on both Windows 7 and earlier Windows. Hopefully we can find a method which is the same size as the old one, since this will greatly simplify the task of fixing old intros to work on Windows 7 (not to mention keeping the size down when adding Windows 7 compatibility).
If all goes well, we will have a fix out before Windows 7 goes public. ;)
@Blueberry
Some thoughts regarding Windows 7 and ntdll.dll issues with older software...
Is it possible to write a 'stub' dll which retrieves a correct address for each api? Such 'transparent' solution will help to run a lot of incompatible software on Vista/W7.
Please think about it.
Quote:
Problem signature:
Problem Event Name: BEX
Application Name: elevated_1280x1024.exe
Application Version: 0.0.0.0
Application Timestamp: 38a66ae8
Fault Module Name: ntdll.dll
Fault Module Version: 6.1.7127.0
Fault Module Timestamp: 4a03d5a1
Exception Offset: 0001e815
Exception Code: c0000409
Exception Data: 00000000
OS Version: 6.1.7127.2.0.0.256.1
Some thoughts regarding Windows 7 and ntdll.dll issues with older software...
Is it possible to write a 'stub' dll which retrieves a correct address for each api? Such 'transparent' solution will help to run a lot of incompatible software on Vista/W7.
Please think about it.
I love the thumbing for tools. I wonder what the rating for bass wouldve been if it was added 10 (?) years ago.
Is tools added to top10? If so, it would mean that all demomakers (ie not at all the majority of pouet users) would thumb up the proper tools and therefor removing themselves from top10.. if pouet where from/by demomakers that is.
The moral is that we never should have any censorship or let the demoscene outreach people govern this site. The majority of pouet.net is not demomakers they are just here because Iam. It's my fansite and you all have accounts to enlarge me.
Thank you very much.
yours,
stefan
Is tools added to top10? If so, it would mean that all demomakers (ie not at all the majority of pouet users) would thumb up the proper tools and therefor removing themselves from top10.. if pouet where from/by demomakers that is.
The moral is that we never should have any censorship or let the demoscene outreach people govern this site. The majority of pouet.net is not demomakers they are just here because Iam. It's my fansite and you all have accounts to enlarge me.
Thank you very much.
yours,
stefan
stfu
See http://www.harmonysecurity.com/blog/2009/06/retrieving-kernel32s-base-address.html - it's only 2 bytes larger.
Also, all programs are called from kernel32:BaseThreadInitThunk, so you can use 11 bytes to get the base:
EntryPoint: ; can be 1 or SectionAlign+1
pop edx ; 'Z'
push edx
FindMZ:
dec edx
cwd ; AX must be set <8000h before
cmp word[edx],'MZ'
jne FindMZ
Also, all programs are called from kernel32:BaseThreadInitThunk, so you can use 11 bytes to get the base:
EntryPoint: ; can be 1 or SectionAlign+1
pop edx ; 'Z'
push edx
FindMZ:
dec edx
cwd ; AX must be set <8000h before
cmp word[edx],'MZ'
jne FindMZ
I wrote a little loader in an attempt to be able to watch intros working under win7. It doesn't work with all, but at least it works with some.
http://www.pouet.net/prod.php?which=53400
http://www.pouet.net/prod.php?which=53400
and the forgotten thumb coming right up (from behind)
rrrola: The method in your link is exactly what we came up with for the new import code (except we use the InLoadOrder list). The article mentions that the method sometimes does not work on Windows 2000. We have identified the exact circumstances under which it works, which is when kernel32 is imported as the first DLL in the executable's import table (so no easy fix for shellcode here).
The second method you mention was used for the first few versions of Crinkler (up to version 0.3). It only works on 32-bit versions of Windows.
The second method you mention was used for the first few versions of Crinkler (up to version 0.3). It only works on 32-bit versions of Windows.
Cool.
That was new to me. You really should write an article about Crinkler's headers - I doubt anyone has tested the whims of various loaders as thoroughly as you. It would make writing custom compressors a lot easier.
So, I guess the days of the lz32.dll import are over now? :)
So, I guess the days of the lz32.dll import are over now? :)
I forgot the thumb in my previous comment, so here it is!
Looking forward to the next version with win7 fix :)
Looking forward to the next version with win7 fix :)
When linking mingw .o files against msvcrt (vc6 one), crinkler is unable to find __alloca. Which seems to be not very sane, as msvcrt should have one. Making all variables static solves this (and I believe this is the right way to make a 4k, however, i've seen quite contradictory compression results with statc/not static vars using gz-dropping on linux).
w23: This might be some special issue with __alloca (it is a sort of weird thing to do anyway - it doesn't seem like the kind of thing that can be implemented by a library without any cooperation with the compiler). If you send us a "faulty" object file (preferably with source), we can have a look at it.
I agree that static allocation (using global variables) is usually the best solution for 4ks. We have often seen reductions in size when making variables global, in particular when doing it for all variables which are used in a particular way (e.g. all COM object handles).
I agree that static allocation (using global variables) is usually the best solution for 4ks. We have often seen reductions in size when making variables global, in particular when doing it for all variables which are used in a particular way (e.g. all COM object handles).
i think __alloca is just an alias for __chkstk (the guard page "make sure to hit every page at least once" mojo), but i'm not certain.
ryg: That makes sense. It also explains why the call goes away when things are allocated statically. Still, if msvcrt.lib has one, Crinkler should be able to find it.
blueberry, not if it's part of the crt startup code that's always linked statically :) lemme check...
Code:
alloca_8: ; 8 byte aligned alloca
_alloca_probe_8 = alloca_8
push ecx
lea ecx, [esp] + 8 ; TOS before entering this function
sub ecx, eax ; New TOS
and ecx, (8 - 1) ; Distance from 8 bit align (align down)
add eax, ecx ; Increase allocation Size
sbb ecx, ecx ; ecx = 0xFFFFFFFF if size wrapped around
or eax, ecx ; cap allocation size on wraparound
pop ecx ; Restore ecx
jmp _chkstk
i think that's the equivalent in current versions of the CRT. so, yeah, __chsktk wrapper taking 8-byte alignment into account.
my exe is still running in the background after i end by esc when using crinkler, i have to end the process in the task manager. common problem?
i always call ExitProcess(0);
For some reason, whatever I try, I can't get crinkler to work on any other COMPMODE than INSTANT. Everything else crashes crinkler when compression should begin. I'm running Windows XP x64 SP2 and using VC++ 2008 Express.
Even with INSTANT I get amazing results, but it'd be awesome to see what the difference would be with say SLOW for instance.
Even with INSTANT I get amazing results, but it'd be awesome to see what the difference would be with say SLOW for instance.
just tested with 1.1 (had been using 1.1a before) and it works fine.
shinmai: Can you send us your object files along with all the lib and dll files used by the intro as they are on your system, please? We have had multiple reports of Crinkler 1.1a crashing, but we have not been able to reproduce it yet. Apparently it only happens on some machines.
just for the record, I had exactly the same problem as shinmai, Windows XP x64 SP2, VC++ 2008 Professional.
I have no clue since I couldn't have gone beyond seeing the title bar of the Crinkler project on Mentor's laptop while fixing Muon Baryon, but it sounds like a 64-bit issue. I might be wrong though.
The tool itself rules, but, any hint about a win7 fix?
( and no, activating windows xp sp3 compatibility in the exe settings won't make it :) )
thx in advance..
( and no, activating windows xp sp3 compatibility in the exe settings won't make it :) )
thx in advance..
I second the Windows 7 (x64) fix. Blueberry: if you need a tester, just mail me.
I'll be willing to test too! I'll be able to run a lot of crippled demos. :D
Specs if yer interested:
Dell Inspiron 1520 running Windows 7 RC1 64 bit Build 7100
Latest official nVidia mobile drivers (186.81 WHQL) and DirectX Drivers
320GB 7,200RPM HDD
4GB RAM
T7250 Core2Duo 2x2.0Ghz
Geforce 8600M GT 256MB
"Code us happy!"
Specs if yer interested:
Dell Inspiron 1520 running Windows 7 RC1 64 bit Build 7100
Latest official nVidia mobile drivers (186.81 WHQL) and DirectX Drivers
320GB 7,200RPM HDD
4GB RAM
T7250 Core2Duo 2x2.0Ghz
Geforce 8600M GT 256MB
"Code us happy!"
The undoubtedly most anticipated Crinkler release is finally here!
We proudly present to you: Crinkler 1.2 !
-Windows 7 compatible.
-Windows 2000 incompatible.
-Slightly smaller header.
-A recompression mode allowing you to update existing crinkler exe-files to the new header and import code.
-A few minor compression tweaks.
Now go recompress those old incompatible intros! :)
We proudly present to you: Crinkler 1.2 !
-Windows 7 compatible.
-Windows 2000 incompatible.
-Slightly smaller header.
-A recompression mode allowing you to update existing crinkler exe-files to the new header and import code.
-A few minor compression tweaks.
Now go recompress those old incompatible intros! :)
Jeg elsker dig som jeg sagde! :D
This is somehow odd. You two really deserve it!
I would like to thumb it up again, just for the effort you guys put in this. Eternal love.
At last!
Thank you very much, guys! I just recompressed the Elevated and it run under Windows 7 like a charm now!
Thank you very much, guys! I just recompressed the Elevated and it run under Windows 7 like a charm now!
mentor: you is a superhelt
also: blueberry.
also: blueberry.
The danes - pants-dropping and compression. Delivered.
yay! :)
You are awesome guys! Keep the good work!
Sensational!
Thanks :)
We got a message while using crinkler v1.2 in VS2005; compilation fails with the following message :
"|-- Estimating models for code ----------------------------|
Project : error PRJ0002 : Error result 3 returned from 'E:\tits-910d\link.exe'."
Getting back on v1.1 resolved the issue.
Still, wonderful tool
"|-- Estimating models for code ----------------------------|
Project : error PRJ0002 : Error result 3 returned from 'E:\tits-910d\link.exe'."
Getting back on v1.1 resolved the issue.
Still, wonderful tool
nice! (thumb up)
great job, too bad i can't thumb it twice :D
Windows 7 support \o/
v1.2 and 1.1a crash for me with any program being compiled! 1.1 works fine. I tested it with VC++ 2008 and Vista 32 and 64 bit. Any guess?
Thanks for all the crash reports. So far we have been unable to reproduce the problem on our own machines (with similar OSes to the ones mentioned in the crash reports), so our strategy will be to try to get access to a crashing machine to test on.
Anyone who wants to contribute towards this goal can do so by bringing a crashing machine to TRSAC 2009, where we will be present and ready to do the debugging.
Anyone who wants to contribute towards this goal can do so by bringing a crashing machine to TRSAC 2009, where we will be present and ready to do the debugging.
Anybody tried /RECOMPRESS+/REPORT combo? Is this a feature or a byproduct? ;)
(also I really miss double-thumbing option)
(also I really miss double-thumbing option)
It is a feature, though currently not a highly developed one. We have some ideas for improving the output in various ways, such as automatically detecting label positions and the names of imported functions.
I get the same crash on XP (HE 2002) SP3, if the info helps at all. VC++ 2005 (yeah I know). Trying to compile an opengl prog with glsl, win32. Works with older crinklers.
Hi Mentor. I have some improvements regarding the decompression header. Could we continue on email?
sure. i believe you got my e-mail at function. if not, just use 'mynick at crinkler net'. I'm excited to hear about your discoveries :)
This sounds so scary that I wonder how the heck you can gain bytes from an already optimized-to-death code.
I've tried producing some extremely low entropy code (99% of it are mov/push/call, all using just eax register), and I have just one question from my side: is there any way of removing import-by-hash code in favor of native system imports (there is not much functions in D3D code anyway) or moving import code to the end of the code section (which would mean one extra jmp, but entropy gains may be worth it)? Maybe I'm totally wrong about it, but I think import code sitting in front of intro code may sometimes setup prediction tables for different coding style, than rest of the intro uses.
Decipher: Certainly rrrola has different definition of optimized-to-death (maybe we should call it optimizing-to-afterlife). ;)
Decipher: Certainly rrrola has different definition of optimized-to-death (maybe we should call it optimizing-to-afterlife). ;)
Do you answer mails lately or shall i paste it here ?
hitch: sorry, i completely forgot about your e-mail. it seems you are not the only one having problems with crinkler 1.2. Blueberry has been looking into the issue, so we will hopefully know something soon. Are you by any chance on an AMD system? :)
Yep.
It seem pretty certain that it's the intel compiler (at least the currently used version) producing crashing code for amd systems.
As i explained in the email it crashes because of an array which isn't initialized properly (null pointer) why would a compiler produce such code ?
I'm on an AMD system as well and I get the "Error result 3 returned..." or a crash as well. Both with VS 2008 RTM and SP1, Windows 7.
My laptop is XP and Intel-based, and on that system everything comes up roses.
Hope this helps you guys fix a great tool, and thanks in advance!
My laptop is XP and Intel-based, and on that system everything comes up roses.
Hope this helps you guys fix a great tool, and thanks in advance!
Is possible to decompress everything from EXE file?
There currently exists no decompressor for Crinkler-compressed files. The closest thing is the /RECOMPRESS option of Crinkler 1.2, which produces a new Crinkler-compressed EXE. If you just want to have a look at the code and data in the file, you can combine /RECOMPRESS with /REPORT to get a disassembly and hexdump of the contents of the EXE.
After i recompiled my 4k with VS2010 and crinkler i notice the exe is much bigger. I check it again with VS2008 and the exe was smaler. Without using crinkler in both VS Studios the exe file was same size. I quick my both report files but i dont have to much time now to find out.
can someone confirm this ????
can someone confirm this ????
Ciclope: the uncompressed file sizes are probably the same because they are aligned to 512 bytes size.
Just make sure no extra bullshit such as the path the the PDB is embedded within the executable. You have to know your compiler as if it's your partner's body. :)
@ Decipher
I have in both VS the same compiler and linker settings. And of course is the same Code. Now i will look the ASM report file from crinkler to find out.
Gargaj: Yes i know what you mean, but i mean the real exe size not the aligned size saved on HD.
I have in both VS the same compiler and linker settings. And of course is the same Code. Now i will look the ASM report file from crinkler to find out.
Gargaj: Yes i know what you mean, but i mean the real exe size not the aligned size saved on HD.
I'm crinkled!
helped me no end in making my first 1k :)
Awesome tool!
I'm using Crinkler for some months. Currently, I'm writing a synth that will probably produce an OBJ and I want to know if Crinkler removes dead code where present or if, on the other side, I should remove it manually before producing the OBJ.
"1>Project : error PRJ0002 : Error result 3 returned from [...]" too .. :(
Win Xp + VC 2008 SP1 & 2010
Win Xp + VC 2008 SP1 & 2010
ciclope, the compiler/linker is also aligning the .exe
merry, shouldn't your linker take care of REAL dead code? (however, neither crinkler, the compiler nor the linker will do runtime analysis to remove your pseudo dead code).
merry, shouldn't your linker take care of REAL dead code? (however, neither crinkler, the compiler nor the linker will do runtime analysis to remove your pseudo dead code).
XT95, same setup, same problem here,
did you find a solution ?
did you find a solution ?
p4p4 -> I saw many people who have this problem, and actually no solution :(
merry: Crinkler removes any function which is not called from live code, if the compiler has placed the function in its own section (Crinkler can only remove whole sections). Check the compression report to make sure the function is gone.
About the crash problem: We are almost certain the problem is with the OpenMP implementation in the Intel compiler. We are investigating alternatives.
One option is to use the Microsoft compiler instead. However, the OpenMP implementation in the Microsoft compiler does not have an option to link statically, which would necessitate an OpenMP DLL alongside Crinkler. Besides being cumbersome (you have to remember the DLL every time you copy Crinkler around), such side-by-side deployment potentially requires some nasty manifest trickery and also violates the distribution license for the MSVC runtime DLLs (they must be installed by an installer).
Does anyone know of any appropriately licensed static library giving equivalent functionality to OpenMP?
One option is to use the Microsoft compiler instead. However, the OpenMP implementation in the Microsoft compiler does not have an option to link statically, which would necessitate an OpenMP DLL alongside Crinkler. Besides being cumbersome (you have to remember the DLL every time you copy Crinkler around), such side-by-side deployment potentially requires some nasty manifest trickery and also violates the distribution license for the MSVC runtime DLLs (they must be installed by an installer).
Does anyone know of any appropriately licensed static library giving equivalent functionality to OpenMP?
We found a usable workaround for AMD systems, until the problem is fixed:
Use the old crinkler 1.1 (which can compress but generates exes that crashes on windows 7) as linker, and then make a post build event that uses crinkler 1.2 /recompress to get the final exe without doing any further compression. That's only 1-2 seconds additional build time for your crinkler builds.
Use the old crinkler 1.1 (which can compress but generates exes that crashes on windows 7) as linker, and then make a post build event that uses crinkler 1.2 /recompress to get the final exe without doing any further compression. That's only 1-2 seconds additional build time for your crinkler builds.
hi, im trying to compress my exe and gets error.
in crinkler 1.1: fatal error: LNK 0: failed to load file 'file.exe';
in 1.2: error: LNK: Unsupported file type.
i have an exe-file. i changed entry point to: mainCRTStartup. using Visual C++ Express 2008 with release build. no manifest. x86. subsystem: windows.
ive tried different command line options without luck.
im running Vista 32bit on laptop w. intel dual cpu. any ideas?
in crinkler 1.1: fatal error: LNK 0: failed to load file 'file.exe';
in 1.2: error: LNK: Unsupported file type.
i have an exe-file. i changed entry point to: mainCRTStartup. using Visual C++ Express 2008 with release build. no manifest. x86. subsystem: windows.
ive tried different command line options without luck.
im running Vista 32bit on laptop w. intel dual cpu. any ideas?
rudi: crinkler replace your linker, it's not something you run on an exe-file...
rudi: Crinkler does not operate on .exe files. Rather, it replaces the linker to operate directly on the .obj and .lib files.
See the INSTALLATION section in the manual for information on how to integrate Crinkler into the Visual Studio build process.
See the INSTALLATION section in the manual for information on how to integrate Crinkler into the Visual Studio build process.
i thought it was an exe-packer all along. :P got it to work. thanks.
Crinkler 1.3 is released!
The long discussed AMD crash bug is now fixed. All that was needed to fix it was to use a newer version of the Intel compiler.
Other highlights include:
- Optimizations to the decompression code reducing the size of the header by 21 bytes. Big thanks to Řrřola for his suggestions leading to these reductions.
- New option - /OVERRIDEALIGNMENTS - to specify label alignments, enabling much higher alignments than what is possible using the normal object file alignment mechanism.
- No limit on the number of calls that can be transformed by call transform (previous limit was 255).
- Various other fixes - see the manual or the web site for the full list.
Enjoy!
The long discussed AMD crash bug is now fixed. All that was needed to fix it was to use a newer version of the Intel compiler.
Other highlights include:
- Optimizations to the decompression code reducing the size of the header by 21 bytes. Big thanks to Řrřola for his suggestions leading to these reductions.
- New option - /OVERRIDEALIGNMENTS - to specify label alignments, enabling much higher alignments than what is possible using the normal object file alignment mechanism.
- No limit on the number of calls that can be transformed by call transform (previous limit was 255).
- Various other fixes - see the manual or the web site for the full list.
Enjoy!
oh thank god for the AMD fix - thanks guys.
Also thanks for fixing the issue with absolute paths!
kringler!
I'd thumb it up again if I could... Thanks Blueberry!
I'll thumb it up for you, RareWtFailWhale. Here you go.
crinkler - making awesome 4ks possible!
.
Thank you very much for the crash bug fix. I have used crinkler for many years now and under a virtual machine because of the crash bug. But now I don't need a virtual machine anymore to run crinkler. Thank you guys!
You are the best!
Hi everyone,
I'm trying to use crlinker tonight for the first time with visual studio 2010 express (simply hello world). I placed crlinker to my project folder and renamed to link.exe. I have strictly followed the installation part in manual.txt. But ... the linking step failed with the error code -1073740777. Is this kind of error was arrived to one of you? Of course my project compile fine if I remove /CRLINKER option.
I'm trying to use crlinker tonight for the first time with visual studio 2010 express (simply hello world). I placed crlinker to my project folder and renamed to link.exe. I have strictly followed the installation part in manual.txt. But ... the linking step failed with the error code -1073740777. Is this kind of error was arrived to one of you? Of course my project compile fine if I remove /CRLINKER option.
"/CRLINKER" ? should be "/CRINKLER" for starters.
Another thing to be sure of is, make sure it's in your solution directory, not the project directory (though they might be the same). Look for the .sln file.
Another thing to be sure of is, make sure it's in your solution directory, not the project directory (though they might be the same). Look for the .sln file.
Oh, hands down the best cruncher around and I haven't even thumbed it up!
Hi Ferris, thanks to answer so quickly.
Sorry for my typing mistake in my previous post. I use /CRINKLER in additionnal link option.
Well, link.exe is in the same folder of my .sln file. I have launched a build without /CRINKLER option only for checking my project with the visual studio linker and it works. With /CRINKLER I got the error.
Sorry for my typing mistake in my previous post. I use /CRINKLER in additionnal link option.
Well, link.exe is in the same folder of my .sln file. I have launched a build without /CRINKLER option only for checking my project with the visual studio linker and it works. With /CRINKLER I got the error.
NeKoFu: This sounds a bit like the error some people were seeing with 1.2 on AMD CPUs. Which Crinkler version are you using? Which CPU do you have?
I can not believe I never placed a comment here. Thank you so much for sharing this truly wonderful tool.
Blueberry: I use the 1.3 on an Intel I5
Thanks for your help
Thanks for your help
I'm having some problems every time I use any componented of the standard template library or some other simple functions (std::string, std::ofstream, exit(0)).
I get this error whenever I use any of the std componentes: LNK: Cannot find symbol '___security_cookie'
If I use exit(0) I get this error: LNK: Cannot find symbol '__imp__exit'
Which libraries am I missing?
I'm using Visual Studio 2010 & Windows 7 64bit (I compile as 32 bit). Is VS2010 and Win7 supported?
I get this error whenever I use any of the std componentes: LNK: Cannot find symbol '___security_cookie'
If I use exit(0) I get this error: LNK: Cannot find symbol '__imp__exit'
Which libraries am I missing?
I'm using Visual Studio 2010 & Windows 7 64bit (I compile as 32 bit). Is VS2010 and Win7 supported?
I managed to fix the problems by linking the following libs:"comsupp.lib" "delayimp.lib" "libcmt.lib" "libcpmt.lib" "libcpmt1.lib" "msvcmrt.lib" "msvcprt.lib" "msvcrt.lib"
"msvcurt.lib" "oldnames.lib" "vcomp.lib" "ptrustu.lib" "RunTmChk.lib" "pgort.lib" "pgobootrun.lib"
:D
"msvcurt.lib" "oldnames.lib" "vcomp.lib" "ptrustu.lib" "RunTmChk.lib" "pgort.lib" "pgobootrun.lib"
:D
To get rid of the ___security_cookie dependency, set Buffer Security Check to No in the Code Generation options.
Use ExitProcess(0) instead of exit(0) to call directly into kernel32.dll and thus avoid this standard library dependency.
Read the section on Standard Runtime Libraries in the manual for further tips.
Use ExitProcess(0) instead of exit(0) to call directly into kernel32.dll and thus avoid this standard library dependency.
Read the section on Standard Runtime Libraries in the manual for further tips.
Replacing exit(0) with ExitProcess(0) did the trick.
Question: I want to use DirectX 11 for graphics, but I always get the ___security_cookie error whenever I try to use any DX11 code. Is it possible for me to use it or would I have to use some older DX9 version for graphics?
Question: I want to use DirectX 11 for graphics, but I always get the ___security_cookie error whenever I try to use any DX11 code. Is it possible for me to use it or would I have to use some older DX9 version for graphics?
I tried some DX11 code, and I can get rid of the ___security_cookie error by setting Configuration Properties -> C/C++ -> Code Generation -> Buffer Security Check to No. Works in both VS 2008 and 2010 here.
If you send your project and source files, we can have a look at it.
If you send your project and source files, we can have a look at it.
NeFoKu: If you send your project, source and object files, we will see if we can reproduce the error.
Oops, haven't thumbed this yet :)
I just got the following message from crinkler 1.3:
I'm not having large arrays or structs inside my function, but it's recursive, which ends up being the same: MSVC thinks you're using (its definition of) lots of stack and ends up generating __chkstk calls.
You can work around this by supplying the /Gs option with a sufficiently large value, such as /Gs999999 (http://support.microsoft.com/kb/100775/en-us).
Perhaps you might want to mention that in crinkler's hint message too.
Quote:
error: LNK: Cannot find symbol '__chkstk'.
* HINT: Avoid declaring large arrays or structs inside functions. Use global variables instead.
I'm not having large arrays or structs inside my function, but it's recursive, which ends up being the same: MSVC thinks you're using (its definition of) lots of stack and ends up generating __chkstk calls.
You can work around this by supplying the /Gs option with a sufficiently large value, such as /Gs999999 (http://support.microsoft.com/kb/100775/en-us).
Perhaps you might want to mention that in crinkler's hint message too.
Oh, actually I AM using lots of local storage, nothing to do with being recursive. Still, /Gs is able to fix it for me.
Late but very deserved thumb :)
I've used it and it rules -> thumb. I only wish that the source code was available..
I just used it to compress the first working release build for windows of an old 4k intro, which I'm using to port my engine from linux.
Crinkler saved *326* bytes so far, and I'm sure that it will be able to save much more with some tweaking!
Crinkler saved *326* bytes so far, and I'm sure that it will be able to save much more with some tweaking!
Is there a way to replace the IBH code and do "some other things"? :)
What Tjoppen said.
What Tjoppen said.
after hand-crafting some linux elf32 executables and gz-dropping them afterwards, i would pretty much like to get this baby ported to linux. how deep is it win32/PE-specific? is it at all possible to get just the bare packing algorithm? i could do the linking stage for elf32 myself
Mandatory thumb. Very impressive material here.
Excellent Cruncher
Is there any update planned? Any idea of when forwarded RVAs will be supported?
Anyway, THE cruncher. The only. Completely amazing piece of work.
Anyway, THE cruncher. The only. Completely amazing piece of work.
merry: We didn't have any plans to support forwarded RVA imports, but if there is a demand for it, we will consider it of course. It will add a considerable amount of code to the importer, though, so you will probably be better off doing things differently if possible. Which function do you need it for?
Messrs C. Rinkler and Sons, bespoke byte removal services
I had a quick look at this today, and I see some things in the import code:
64678B4730 mov eax,fs:[bx+30]
could be
6467A13000 mov eax,fs:[30]
(maybe a zero in the instruction helps somehow)
mov ebx,<image base>
could be
mov ebx,[eax+08]
if placed after the fs: line, for a 2 bytes saving.
also, the check for DLL loading:
test ebp,ebp
jne OUT
push 00000000
push 00000000
push edx
push 00000000
could be
test ebp,ebp
jne OUT
push ebp
push ebp
push edx
push ebp
since ebp is known to be zero at that point, for a 3 bytes saving.
the code in the header could be shortened by at least 3 bytes, but I didn't see any advantage because it seems that nothing can move into the gap.
64678B4730 mov eax,fs:[bx+30]
could be
6467A13000 mov eax,fs:[30]
(maybe a zero in the instruction helps somehow)
mov ebx,<image base>
could be
mov ebx,[eax+08]
if placed after the fs: line, for a 2 bytes saving.
also, the check for DLL loading:
test ebp,ebp
jne OUT
push 00000000
push 00000000
push edx
push 00000000
could be
test ebp,ebp
jne OUT
push ebp
push ebp
push edx
push ebp
since ebp is known to be zero at that point, for a 3 bytes saving.
the code in the header could be shortened by at least 3 bytes, but I didn't see any advantage because it seems that nothing can move into the gap.
even better: ebx=fs:[30] on process start (it's even documented somewhere), so push ebx in the header, pop eax in the second stage, no need for the fs: line at all.
mov eax,[edx+1C]
add eax,ebp
mov eax,[eax+4*ecx]
mov [esp+1C],eax
if ebp and edx were exchanged, you could use
add edx,[ebp+1C]
mov eax,[edx+4*ecx]
mov [esp+1C],eax
for a 2 bytes saving
extending that:
mov eax,[edx+4*ecx]
mov [esp+1C],eax
popad
could be
pop eax
push [edx+4*ecx]
popad
for a 3 bytes saving.
of course, the actual savings after compressing these might be 0 or worse. :-/
add eax,ebp
mov eax,[eax+4*ecx]
mov [esp+1C],eax
if ebp and edx were exchanged, you could use
add edx,[ebp+1C]
mov eax,[edx+4*ecx]
mov [esp+1C],eax
for a 2 bytes saving
extending that:
mov eax,[edx+4*ecx]
mov [esp+1C],eax
popad
could be
pop eax
push [edx+4*ecx]
popad
for a 3 bytes saving.
of course, the actual savings after compressing these might be 0 or worse. :-/
the original PEB_LDR_DATA code could be used, if you resolved LoadLibraryExA instead of LoadLibraryA. the ExA version exists in kernelbase.dll and kernel32.dll, and it takes two additional parameters which would both be zero.
for the import table on Win2k, the requirement was to import from either kernel32.dll, or something that imports from kernel32.dll, so that kernel32.dll is loaded somehow. lz32.dll could have been wmi.dll for a 1 byte saving.
for the import table on Win2k, the requirement was to import from either kernel32.dll, or something that imports from kernel32.dll, so that kernel32.dll is loaded somehow. lz32.dll could have been wmi.dll for a 1 byte saving.
qkumba:
Thanks for all the input.
It seems you are right about ebx=[fs:30] on startup. A quick test suggests that we can
save about 3 bytes. It does complicate the call transform code, but I think it will still be a net win in that case.
Can we rely on this across all windows versions?
It would be much appreciated if you can find the documentation you are mentioning :)
This seems to be slightly worse, at least for my example project, because we go from 2 to 3 non-zero bytes.
Optimizing for compressed size is tricky.
We reuse the same instruction sequences, addressing modes and registers as much as possible.
In the end it tends to compress better than the more compact alternatives we have tried. (lodsd+xchg+SIB)
This is of course highly context dependent, so you would ideally want the import code and
intro code to be written in a similar style.
You do realize that these pushes are in your code, right?
Yes, ebp is guaranteed to be zero after the import code, but you will have to exploit it
yourself. We are not going to rewrite user code :)
Yes, gaining a lone byte somewhere in the header doesn't really help.
I'm guessing part of the saving you are mentioning is in the code around the stack reserve field.
You also need to be aware that the code inside the header is not as naive as it might seem, as it is under some additional constraints.
mov ebx, dword 3
is actually just a shorter way of jumping across the subsystem field (dword 2/3), while at the same time initializing ebx to something >1.
The next 4 fields are the reserve/commit fields for the stack and heap. We have to be extra careful about these, as they need to be small
valued dwords in order for windows not to explode, so the instructions are chosen to always have 00/01 in the most significant bytes of these 4 dwords :)
I tried this and it seems to be more than a byte worse than the original code.
mov eax, [edx+xxh]
add eax, ebp
This is actually one of these repeated patterns I mentioned earlier. This is the third instance of the pattern and the fourth instance of add eax, ebp , so at this point it is at a significant discount :)
I also prefer to use ebp instead of edx as it is preserved across calls, so it is easier to exploit it being 0 in your intro.
Wouldn't this store the value into EDI instead of EAX?
This is from the old header. We no longer support win2k. Also, you wouldn't really save 1 byte as it was in a 8 byte slot in the header.
This sounds very interesting. I'm not really sure I follow. How do you get to kernelbase.dll and can you do this reliably on all windows versions?
Using kernelbase would result in an overhead when the intro imports from kernel32, but I guess we could redirect the imports to kernelbase in the
instances where we can. Either way, I'm curious about this :)
Thanks for all the input.
It seems you are right about ebx=[fs:30] on startup. A quick test suggests that we can
save about 3 bytes. It does complicate the call transform code, but I think it will still be a net win in that case.
Can we rely on this across all windows versions?
It would be much appreciated if you can find the documentation you are mentioning :)
Quote:
mov ebx,<image base>
could be
mov ebx,[eax+08]
if placed after the fs: line, for a 2 bytes saving.
This seems to be slightly worse, at least for my example project, because we go from 2 to 3 non-zero bytes.
Optimizing for compressed size is tricky.
We reuse the same instruction sequences, addressing modes and registers as much as possible.
In the end it tends to compress better than the more compact alternatives we have tried. (lodsd+xchg+SIB)
This is of course highly context dependent, so you would ideally want the import code and
intro code to be written in a similar style.
Quote:
also, the check for DLL loading:
test ebp,ebp
jne OUT
push 00000000
push 00000000
push edx
push 00000000
could be
test ebp,ebp
jne OUT
push ebp
push ebp
push edx
push ebp
since ebp is known to be zero at that point, for a 3 bytes saving.
You do realize that these pushes are in your code, right?
Yes, ebp is guaranteed to be zero after the import code, but you will have to exploit it
yourself. We are not going to rewrite user code :)
Quote:
the code in the header could be shortened by at least 3 bytes, but I didn't see any advantage because it seems that nothing can move into the gap.
Yes, gaining a lone byte somewhere in the header doesn't really help.
I'm guessing part of the saving you are mentioning is in the code around the stack reserve field.
You also need to be aware that the code inside the header is not as naive as it might seem, as it is under some additional constraints.
mov ebx, dword 3
is actually just a shorter way of jumping across the subsystem field (dword 2/3), while at the same time initializing ebx to something >1.
The next 4 fields are the reserve/commit fields for the stack and heap. We have to be extra careful about these, as they need to be small
valued dwords in order for windows not to explode, so the instructions are chosen to always have 00/01 in the most significant bytes of these 4 dwords :)
Quote:
mov eax,[edx+1C]
add eax,ebp
mov eax,[eax+4*ecx]
mov [esp+1C],eax
if ebp and edx were exchanged, you could use
add edx,[ebp+1C]
mov eax,[edx+4*ecx]
mov [esp+1C],eax
for a 2 bytes saving
I tried this and it seems to be more than a byte worse than the original code.
mov eax, [edx+xxh]
add eax, ebp
This is actually one of these repeated patterns I mentioned earlier. This is the third instance of the pattern and the fourth instance of add eax, ebp , so at this point it is at a significant discount :)
I also prefer to use ebp instead of edx as it is preserved across calls, so it is easier to exploit it being 0 in your intro.
Quote:
mov eax,[edx+4*ecx]
mov [esp+1C],eax
popad
could be
pop eax
push [edx+4*ecx]
popad
for a 3 bytes saving.
Wouldn't this store the value into EDI instead of EAX?
Quote:
for the import table on Win2k, the requirement was to import from either kernel32.dll, or something that imports from kernel32.dll, so that kernel32.dll is loaded somehow. lz32.dll could have been wmi.dll for a 1 byte saving.
This is from the old header. We no longer support win2k. Also, you wouldn't really save 1 byte as it was in a 8 byte slot in the header.
Quote:
the original PEB_LDR_DATA code could be used, if you resolved LoadLibraryExA instead of LoadLibraryA. the ExA version exists in kernelbase.dll and kernel32.dll, and it takes two additional parameters which would both be zero.
This sounds very interesting. I'm not really sure I follow. How do you get to kernelbase.dll and can you do this reliably on all windows versions?
Using kernelbase would result in an overhead when the intro imports from kernel32, but I guess we could redirect the imports to kernelbase in the
instances where we can. Either way, I'm curious about this :)
Quote:
It seems you are right about ebx=[fs:30] on startup. A quick test suggests that we can
save about 3 bytes. It does complicate the call transform code, but I think it will still be a net win in that case.
Can we rely on this across all windows versions?
It would be much appreciated if you can find the documentation you are mentioning :)
Yes, you can rely on that across all Windows versions. I will look for the documentation.
Quote:
...
You do realize that these pushes are in your code, right?
That was taken from the one file that I examined. It never occurred to me to check which parts Crinkler added. :-)
Quote:
Quote:the code in the header could be shortened by at least 3 bytes, but I didn't see any advantage because it seems that nothing can move into the gap.
Yes, gaining a lone byte somewhere in the header doesn't really help.
I'm guessing part of the saving you are mentioning is in the code around the stack reserve field.
You also need to be aware that the code inside the header is not as naive as it might seem, as it is under some additional constraints.
I understand that. I have some experience in crushing PE headers (see pferrie.host22.com). ;-)
Speaking of which, you don't need any section table at all. It is legal to set NumberOfSections to 0 since you import nothing (import table must be in a section since WinXP), and you still allocate all the memory that you want. DEP is not a problem if section alignment < 4096. I'm sure that complicates things for you, but just in case you didn't know.
Quote:
mov ebx, dword 3
is actually just a shorter way of jumping across the subsystem field (dword 2/3), while at the same time initializing ebx to something >1.
Yes, but as noted above, it's already non-zero.
Quote:
Quote:
for the import table on Win2k, the requirement was to import from either kernel32.dll, or something that imports from kernel32.dll, so that kernel32.dll is loaded somehow. lz32.dll could have been wmi.dll for a 1 byte saving.
This is from the old header. We no longer support win2k. Also, you wouldn't really save 1 byte as it was in a 8 byte slot in the header.
If you moved lfanew to 0x10, for example, then it would fit at offset 2, with room for an instruction or two.
Quote:
Quote:the original PEB_LDR_DATA code could be used, if you resolved LoadLibraryExA instead of LoadLibraryA. the ExA version exists in kernelbase.dll and kernel32.dll, and it takes two additional parameters which would both be zero.
This sounds very interesting. I'm not really sure I follow. How do you get to kernelbase.dll and can you do this reliably on all windows versions?
Using kernelbase would result in an overhead when the intro imports from kernel32, but I guess we could redirect the imports to kernelbase in the
instances where we can. Either way, I'm curious about this :)
When run on Win7, the old PEB code would fetch kernelbase.dll instead of kernel32.dll, and then call the wrong API because the hash loop exited.
So, if you resolved LoadLibraryExA instead of LoadLibraryA, then you'd get a valid address no matter which platform was used.
I haven't seen any demos using kernel32.dll, but I suppose that if one did, then you'd still have a regular import entry for it, so the behaviour would not change visibly. I will test that.
Quote:
You do realize that these pushes are in your code, right?
I just checked again. It is Crinker code. It's the exit code when a DLL can't be loaded.
The "OUT" label that I referenced was the truncated process name (OUT:some address).
ah ok, I misunderstood your suggestion then as ebp is also zero at the end of the normal execution. I'm running with /UNSAFEIMPORT here, so I didn't think of the error message path. Yes, that push 0 can be turned into push ebp. The difference tends to be around +-1byte depending on the style of the intro code. We are using 0 right now, as we have an expectation that sequences of
push 0 is more common than sequences of push ebp in intro code. But looking at it again now it seems to be pretty even with the ebp variant, so we should probably make it a majority vote based on our test suite of intros.
It is very common for 4ks to call ExitProcess, but that wouldn't really be a problem, as I can see it is also in kernelbase.
I tried the old import code again and it is just around one byte smaller after compression. Which will then unfortunately be negated by the need to push two zeroes on the stack before LoadLibraryExA.
Thanks again, It's great to have someone with in-depth knowledge of the darker corners of win32 look into this :)
push 0 is more common than sequences of push ebp in intro code. But looking at it again now it seems to be pretty even with the ebp variant, so we should probably make it a majority vote based on our test suite of intros.
It is very common for 4ks to call ExitProcess, but that wouldn't really be a problem, as I can see it is also in kernelbase.
I tried the old import code again and it is just around one byte smaller after compression. Which will then unfortunately be negated by the need to push two zeroes on the stack before LoadLibraryExA.
Thanks again, It's great to have someone with in-depth knowledge of the darker corners of win32 look into this :)
the LoadLibrary pushes might be achieved in the header for no cost at all...
that would require more than just two pushes as loadlibrary is called in a loop. we actually need two pushes per dll. this can of course still be done, but it will be at least at couple of instructions and won't be easy to fit into the current header layout and we are still talking about a potential 1 byte improvement :)
I'm curious now. If we are guaranteed that ebx=[fs:30] on all windows versions, do we have similar guarantees about any of the other registers, flags, etc.?
I get ecx=esi=edi=0 and edx=eip, but can I rely on any of this? I have tried, and failed, to find this information many times.
I'm curious now. If we are guaranteed that ebx=[fs:30] on all windows versions, do we have similar guarantees about any of the other registers, flags, etc.?
I get ecx=esi=edi=0 and edx=eip, but can I rely on any of this? I have tried, and failed, to find this information many times.
okay, so forget the LoadLibrary thing. :-)
the dec al/inc al part - these seem less likely to appear than dec eax/inc eax.
would it be better to xor eax,eax before the lodsb, and then dec eax and inc eax instead?
the xor/lodsb sequence appears earlier, so maybe that helps compression.
and in that case, perhaps instead of dec/jns, you could use test/jne, and instead of the inc/je, you could use cmp eax,ebp. it's bigger, but might compress better.
you've reached the end of the list when ebp is zero.
as far as initial registers, only ebx is defined. edx=eip only if syscall mode is used (which can be disabled in some ways), other register values have changed over time.
the dec al/inc al part - these seem less likely to appear than dec eax/inc eax.
would it be better to xor eax,eax before the lodsb, and then dec eax and inc eax instead?
the xor/lodsb sequence appears earlier, so maybe that helps compression.
and in that case, perhaps instead of dec/jns, you could use test/jne, and instead of the inc/je, you could use cmp eax,ebp. it's bigger, but might compress better.
you've reached the end of the list when ebp is zero.
as far as initial registers, only ebx is defined. edx=eip only if syscall mode is used (which can be disabled in some ways), other register values have changed over time.
mentor, if you accept mail from Gmail, I have sent you a new PE header code.
still incredibly useful
Any current developments? :)
I would love to have some extra bytes.
I would love to have some extra bytes.
Obviously a thumb up.
It would be great if packing code were released separately, as a lib, together with a possibly unoptimal example unpacker.
Not sure why I haven't thumbed this yet. Big thumb up! :)
Hi, i'm trying to learn dx11 & crinkler
a function call to
D3DX11CompileFromFile(L"shaders.hlsl", 0, 0, "VShader", "vs_5_0", 0, 0, 0, &VS, 0, 0);
reports a linking error LNK: Cannot find symbol '_D3DX11CompileFromFileW@44'
am i missing a .lib or something?
a function call to
D3DX11CompileFromFile(L"shaders.hlsl", 0, 0, "VShader", "vs_5_0", 0, 0, 0, &VS, 0, 0);
reports a linking error LNK: Cannot find symbol '_D3DX11CompileFromFileW@44'
am i missing a .lib or something?
4k intros usually don't load files from disk..
bstrr: i'll try to embed the shader to .exe once i get this tutorial working :)
Anyone else having problems running crinkler intros on win7 x64 nowadays? Apparently something weird is going on, all of them crash instantly for me, even the supposedly win7 fixed Elevated. I don't know whether it's only my machine, or it's due to a recent windows update or something else. Common crash details are below, if this helps anything:
One thing is sure, I had the same OS installation a year ago and they were still working back then.
Code:
Problem Event Name: APPCRASH
Fault Module Name: ntdll.dll
Fault Module Version: 6.1.7601.17725
Exception Code: c0000005
Exception Offset: 000300e2
OS Version: 6.1.7601.2.1.0.256.1
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789
One thing is sure, I had the same OS installation a year ago and they were still working back then.
my code starts at "int WINAPI WinMain(.." if i change it & linker->advanced->entry point, the program won
't run correctly and a zombie program remains in the system. i'm trying to get rid of msvcrt.lib. whenever i remove it from linker dependencies the crinkler won't find the entry point, and if i try to change it the way i do, i get this zombie. help please.
't run correctly and a zombie program remains in the system. i'm trying to get rid of msvcrt.lib. whenever i remove it from linker dependencies the crinkler won't find the entry point, and if i try to change it the way i do, i get this zombie. help please.
I too have many crash with win7 fixed intros (crinkler 1.2 and 1.3). Only one seem to work well after recompress : Lunaquatic.
The issue seems to be an incompatibility with the Geforce R304 driver (from 304.48 to 306.97). The 301.42 is the last one who can properly run crinkler compressed intros.
Any fix planned?
The issue seems to be an incompatibility with the Geforce R304 driver (from 304.48 to 306.97). The 301.42 is the last one who can properly run crinkler compressed intros.
Any fix planned?
Yes, I am using the 306.23 driver myself. Maybe it's related to Windows 8 compatibility changes?
Thank you for the bug reports, and for the pointer to the GeForce driver. Now we have something to investigate. I have seen some problems on one Windows 7 machine myself, and it is also running a driver in that version range.
Crinkler hangs with following code: (full source)
http://www.sistavip.com/pub/CrinklerPackage.zip
Pls Hlp! (We all love Crinkler) -> Kirill
http://www.sistavip.com/pub/CrinklerPackage.zip
Pls Hlp! (We all love Crinkler) -> Kirill
nvinit.dll has a known bug in it that causes crashes like this. See my comment on the page that las linked. If that's the same crash that people are seeing (and if the solution is the same), then it's not Crinkler's fault.
I went back to nvidia driver 301.42, it worked of course. But then I installed 306.97 again, and now it works. Even after reboot ... now somebody explain THIS! :-)
By the way, thanks for all the info, guys!
By the way, thanks for all the info, guys!
Question about the Crinkler license...
It says "You may use Crinkler for any non-commercial purpose". May Crinkler be used for commercial purposes as well? If so, any restrictions besides "must not be used for any safety critical purpose"?
Specifically, I have a tiny two-option config utility. I've got to fit it into 4KB somehow.
Thanks
James
It says "You may use Crinkler for any non-commercial purpose". May Crinkler be used for commercial purposes as well? If so, any restrictions besides "must not be used for any safety critical purpose"?
Specifically, I have a tiny two-option config utility. I've got to fit it into 4KB somehow.
Thanks
James
question about crinkler, can I use crinkler to compile Chevrolet : http://www.pouet.net/prod.php?which=60469
???
Quote:
May Crinkler be used for commercial purposes as well?
No, Crinkler may not be used for commercial purposes. That could indeed be spelled out more clearly in the license.
Try leaving out the manifest, linking with the VS6 version of msvcrt.lib (see the Crinkler manual for some guidelines on this) and stripping the executable. This can usually bring the overhead down to 1k (512 bytes header + 512 bytes import table), which means you will have 3k for your code, which should be plenty for what you describe.
Quote:
Cannot find symbol '_D3DX11CompileFromFileW@44'
Try d3dx11.lib from the DirectX SDK.
That is a pity. Is there no commercially licensable version, then, if not the standard one?
Basically, Crinkler is an experimental hack, and executables produced by it will likely break on some future Windows version and/or graphics driver (until recompressed using a then-updated version of Crinkler). That is why we consider it unsuitable for commercial uses.
Furthermore, we don't expect any commercial uses of cramming as much as excruciatingly possible into 4 kilobytes, though we may of course yet be surprised in this regard.
It still sounds to me like the utility you are making can be comfortably squeezed into 4k by using standard exe size reducing tricks, most of which you would need to do anyway to make Crinkler happy. A few more tricks for the list:
- Define your own entry point (/ENTRY option to the VS linker).
- Define your own MS-DOS stub (/STUB option to the VS linker). Give it the first 64 bytes of any exe file as the stub file to produce an empty stub. This should make the whole header fit in 512 bytes with up to 3 sections (code, rdata (also containing import table) and bss).
- Do not include reloc information (/FIXED option to the VS linker).
Furthermore, we don't expect any commercial uses of cramming as much as excruciatingly possible into 4 kilobytes, though we may of course yet be surprised in this regard.
It still sounds to me like the utility you are making can be comfortably squeezed into 4k by using standard exe size reducing tricks, most of which you would need to do anyway to make Crinkler happy. A few more tricks for the list:
- Define your own entry point (/ENTRY option to the VS linker).
- Define your own MS-DOS stub (/STUB option to the VS linker). Give it the first 64 bytes of any exe file as the stub file to produce an empty stub. This should make the whole header fit in 512 bytes with up to 3 sections (code, rdata (also containing import table) and bss).
- Do not include reloc information (/FIXED option to the VS linker).
It has been far too long, but now the wait is over. Another major release of Crinkler has seen the light of day. Crinkler 1.4 is out!
Major new features include:
- The crash problem seen with recent NVIDIA drivers is fixed. Thanks to qkumba for pointing to the source of the problem: the export table pointer. This is now zero, and all is well.
- Completely redesigned header, using zero sections, a smaller PE header offset and cramming every last bit of unused space in the header (and a fair portion of the used space as well) with code. The result is a size reduction of typically 30-50 bytes for your intro. Again thanks to qkumba for guidance concerning this header layout.
- Forwarded RVA imports are now supported. This for instance enables Crinkler to link with uFMOD or libv2 out of the box.
- Dynamic C++ initializers (initializers for global variables) are supported. Direct calls to all initializer functions are inserted before the entry point. They can be disabled using the /NOINITIALIZERS switch.
- Support for setting the Large Address Aware flag on the output executable. This enables the program to allocate more than 2GB of memory (usually around 3.5GB on 64-bit systems). It is still still not possible to allocate more than 2GB statically, as this is the maximum virtual size of 32-bit Windows executables.
- Crinkler reports all unresolved symbols, rather than stopping after the first one, and indicates the object file referencing each symbol.
See the manual and the website for the full list.
Go pick it up at crinkler.net today! Enjoy! :-D
Major new features include:
- The crash problem seen with recent NVIDIA drivers is fixed. Thanks to qkumba for pointing to the source of the problem: the export table pointer. This is now zero, and all is well.
- Completely redesigned header, using zero sections, a smaller PE header offset and cramming every last bit of unused space in the header (and a fair portion of the used space as well) with code. The result is a size reduction of typically 30-50 bytes for your intro. Again thanks to qkumba for guidance concerning this header layout.
- Forwarded RVA imports are now supported. This for instance enables Crinkler to link with uFMOD or libv2 out of the box.
- Dynamic C++ initializers (initializers for global variables) are supported. Direct calls to all initializer functions are inserted before the entry point. They can be disabled using the /NOINITIALIZERS switch.
- Support for setting the Large Address Aware flag on the output executable. This enables the program to allocate more than 2GB of memory (usually around 3.5GB on 64-bit systems). It is still still not possible to allocate more than 2GB statically, as this is the maximum virtual size of 32-bit Windows executables.
- Crinkler reports all unresolved symbols, rather than stopping after the first one, and indicates the object file referencing each symbol.
See the manual and the website for the full list.
Go pick it up at crinkler.net today! Enjoy! :-D
Nice update! 
Thanks for your hard work on this. :)
.
Fantastic guys can finally get the 4k's working on my Windows 8 lappy!
Excellent, fantastic tool. Many thanks for the update!
yay, this update is awesome news!! thanks! and even more awesome is that it means you are working in a new 4k too?
iq: no, just crinkler for now :)
Thumb up! Also for à new IQ&Mentor&Puryx 4kb ;))
@mentor: ohhhh :(
Key-Real:
Actually, it doesn't hang, it just takes very long to get started because of the large amount of code. Given enough time, Crinkler 1.3 runs out of memory and crashes. :-/
The new version can handle more memory - enough to successfully compress your example code. Try it! :-D
Quote:
Crinkler hangs with following code: (full source)
http://www.sistavip.com/pub/CrinklerPackage.zip
Actually, it doesn't hang, it just takes very long to get started because of the large amount of code. Given enough time, Crinkler 1.3 runs out of memory and crashes. :-/
The new version can handle more memory - enough to successfully compress your example code. Try it! :-D
Hey! I had the idea of trying to gain a few bytes by putting every function in its own text section, and to let crinkler rearrange my code for optimal order for compression.
When I tried this out, I got a size increase overall due to 4-byte section alignment, and NOPs being inserted as padding.
I tried just putting one function in a separate section and let crinkler re-arrange it and gained 5 bytes, and then 2 additional bytes when I re-arranged the code manually (and let everything still be in one text section), so there may be potential.
So, the idea is to remove padding NOPs (0x90) from text sections. I have no idea if this would be easy to implement for you though (the HTML report showing disassembled code is promising, at least).
I guess you could theoretically do the same with data sections, but there may not be any way to detect this. Or well, I haven't checked, perhaps data sections are padded with 0x00?
When I tried this out, I got a size increase overall due to 4-byte section alignment, and NOPs being inserted as padding.
I tried just putting one function in a separate section and let crinkler re-arrange it and gained 5 bytes, and then 2 additional bytes when I re-arranged the code manually (and let everything still be in one text section), so there may be potential.
So, the idea is to remove padding NOPs (0x90) from text sections. I have no idea if this would be easy to implement for you though (the HTML report showing disassembled code is promising, at least).
I guess you could theoretically do the same with data sections, but there may not be any way to detect this. Or well, I haven't checked, perhaps data sections are padded with 0x00?
Last but not least, thanks for an absolutely amazing tool! I'll have to buy you two beer, dinner or whatever sometime.
An alternative idea, which wouldn't require the programmer to manually put stuff into separate sections, is to split code sections after jmp and ret instructions prior to reordering in crinkler. This would require an extensive rewrite of all addresses in the code though.
And hmm, I just realized that this may require relative addresses to be modified to be absolute.... this is becoming complex :)
And hmm, I just realized that this may require relative addresses to be modified to be absolute.... this is becoming complex :)
Forcing the alignments of code sections to 1 is actually already on our to-do list, it just didn't make it to the 1.4 release. :)
We have discussed the possibility of chopping up sections based on some kind of code control flow analysis. It would be an experimental feature at best. And even if it would work, I don't think the benefit would be particularly great. If you really need your code split up, you can do so by hand.
We have discussed the possibility of chopping up sections based on some kind of code control flow analysis. It would be an experimental feature at best. And even if it would work, I don't think the benefit would be particularly great. If you really need your code split up, you can do so by hand.
Alright, if I get some time for it I'll try more manual rearrangement and see how much more I can gain in my case.
But yeah, I agree that it may not be worth the effort.
But yeah, I agree that it may not be worth the effort.
Hey Mentor and/or Blueberry,
I'm trying to make a 4K in VS2012 Express with the XP-compatible linker flag (platform toolset = Visual Studio 2012 - Windows XP (v110_xp), instead of the default Visual Studio 2012 (v110) ), and crinkler 1.4 does not like that:
1>LINK : error : cannot parse token '/SUBSYSTEM:WINDOWS,5.01': unknown argument WINDOWS,5.01
Would it be possible to fix this, preferably before Revision?
I'm trying to make a 4K in VS2012 Express with the XP-compatible linker flag (platform toolset = Visual Studio 2012 - Windows XP (v110_xp), instead of the default Visual Studio 2012 (v110) ), and crinkler 1.4 does not like that:
1>LINK : error : cannot parse token '/SUBSYSTEM:WINDOWS,5.01': unknown argument WINDOWS,5.01
Would it be possible to fix this, preferably before Revision?
To sum up the conversations I had with Seven about this XP compatibility issue:
The fix is to not select the XP compatibility option. Executables created using Crinkler will be XP compatible without it. We will make Crinkler more tolerant to the syntax of this option in a future version.
The fix is to not select the XP compatibility option. Executables created using Crinkler will be XP compatible without it. We will make Crinkler more tolerant to the syntax of this option in a future version.
How to use crinkler in Dev-C++ with TDM-GCC x64 ? Pleaaaase
sandeeeeeh: Crinkler is a 32-bit linker. Compile your code for 32 bits and you should be fine. If you really need 64-bit code, you are out of luck, unfortunately.
You will probably need to link with some MinGW-specific runtime library. To minimize the amount of code pulled in from the runtime library, be sure to define your own entry point (using the /ENTRY option).
Let us know if you still encounter problems.
You will probably need to link with some MinGW-specific runtime library. To minimize the amount of code pulled in from the runtime library, be sure to define your own entry point (using the /ENTRY option).
Let us know if you still encounter problems.
My building try has failed like this.
uncompressed size of code: 208209
uncompressed size of data: 15431
|-- Estimating models for code -----------------------------------------|
This is the end of testing. I can't see no more line.
I want to know the reason of normal exit.
Could you please show me the way?
uncompressed size of code: 208209
uncompressed size of data: 15431
|-- Estimating models for code -----------------------------------------|
This is the end of testing. I can't see no more line.
I want to know the reason of normal exit.
Could you please show me the way?
hjohn: Your code section is quite huge. Crinkler is probably running out of memory.
Does your code section contain one or more big blocks of zeros? If so, try to separate those out into a bss section.
If you actually have 200kb of ordinary code, it will probably not compress down to anywhere near 4kb, and so Crinkler is maybe not the right tool for the job.
If your code is generated and highly redundant, and you expect it to compress down to a few kb, try to generate it at runtime instead.
Does your code section contain one or more big blocks of zeros? If so, try to separate those out into a bss section.
If you actually have 200kb of ordinary code, it will probably not compress down to anywhere near 4kb, and so Crinkler is maybe not the right tool for the job.
If your code is generated and highly redundant, and you expect it to compress down to a few kb, try to generate it at runtime instead.
Hmm, i just migrated an intro from v1.2 to v1.4.
v1.3 brought "Header size reduced by 21 bytes."
v1.4 brought "New zero-section header layout saving around 30-50 bytes."
So i should get an about 50-70bytes smaller .exe in the end, right?
Too bad, i got an .exe 25 bytes bigger. Exceeding the 4096b this way.
Any Idea why this could happen? Will have a look into the manual again now, if i find a clue or a fix i´ll report back here asap.
v1.3 brought "Header size reduced by 21 bytes."
v1.4 brought "New zero-section header layout saving around 30-50 bytes."
So i should get an about 50-70bytes smaller .exe in the end, right?
Too bad, i got an .exe 25 bytes bigger. Exceeding the 4096b this way.
Any Idea why this could happen? Will have a look into the manual again now, if i find a clue or a fix i´ll report back here asap.
You probably have some C++ initializers that weren't being run before. Try /NOINITIALIZERS.
Some intros (Sincere and Atrium) crash with Windows 8.1 and Intel HD Graphics (IronLake) 8.15.10.2900. Of course, these intros are recompressed with crinkler 1.4. All the other 4k intros that I have are working properly.
On the same machine with Windows 8, Sincere worked properly. Unfortunately, there is only one version of this driver for Windows 8.1, so I cannot do other tests on this point.
On the same machine with Windows 8, Sincere worked properly. Unfortunately, there is only one version of this driver for Windows 8.1, so I cannot do other tests on this point.
Blueberry:
Thanks, that did the trick! Have been completely unaware of this switch so far! Damn me!
Thanks, that did the trick! Have been completely unaware of this switch so far! Damn me!
This is one of the greatest gifts to sizecoders and the scene. Thank you.
Any way to save decompressed file, please?
Short answer: There is no such thing as a decompressed file, as there was no uncompressed file produced to begin with.
Long answer: It should be theoretically possible to produce an uncompressed executable from a compressed one. The process would include at least these steps (on top of the actual decompression, which is already there):
- parse up the import data
- find the imported functions in the imported DLLs based on the function hashes
- generate an ordinary import table importing these functions
- split up the code/data/bss section into an initialized and uninitialized part (to avoid producing a file of several hundred megabytes)
- locate the entry point right after the Crinkler import code (and call transformation code, if present)
- produce an ordinary PE header binding all these things together
We don't have any immediate plans to implement this, but we will certainly consider it if there is a good use case for it. What do you need it for?
Long answer: It should be theoretically possible to produce an uncompressed executable from a compressed one. The process would include at least these steps (on top of the actual decompression, which is already there):
- parse up the import data
- find the imported functions in the imported DLLs based on the function hashes
- generate an ordinary import table importing these functions
- split up the code/data/bss section into an initialized and uninitialized part (to avoid producing a file of several hundred megabytes)
- locate the entry point right after the Crinkler import code (and call transformation code, if present)
- produce an ordinary PE header binding all these things together
We don't have any immediate plans to implement this, but we will certainly consider it if there is a good use case for it. What do you need it for?
Magic.
Any chance to do the following with crinkler?
and
See here and here.
Would be really nice to have (opt in) since crinkler seems to be the 8k standard compressor and you have the size for stupid things like that in 8k.
Code:
extern "C" {
_declspec(dllexport) DWORD NvOptimusEnablement = 0x00000001;
}
and
Code:
extern "C"
{
__declspec(dllexport) int AmdPowerXpressRequestHighPerformance = 1;
}
See here and here.
Would be really nice to have (opt in) since crinkler seems to be the 8k standard compressor and you have the size for stupid things like that in 8k.
Le thumb. Thank you for this excellent tool.
las: The Export RVA is currently zero (specifically to prevent the graphics driver from crashing when it goes looking for this export), and it is not used by the decompression code (in contrast with some other "meaningful" header fields).
So yes, it should be quite possible to add an export table to a Crinkler executable. It can even be easily added to an existing executable, or during a recompression, so you can look at the size of your intro and then decide to add it if there is space enough.
The overhead will probably be around 50-60 bytes plus the lengths of the exported names. But as you say, this is probably something you could often spare in an 8k.
The big question is of course whether whoever wants to read the export table can understand the contorted zero-section layout of a Crinkler executable. But my guess is that the graphics drivers merely let the normal loader do its job and then go and look in the loaded image. So that should work fine.
So yes, it should be quite possible to add an export table to a Crinkler executable. It can even be easily added to an existing executable, or during a recompression, so you can look at the size of your intro and then decide to add it if there is space enough.
The overhead will probably be around 50-60 bytes plus the lengths of the exported names. But as you say, this is probably something you could often spare in an 8k.
The big question is of course whether whoever wants to read the export table can understand the contorted zero-section layout of a Crinkler executable. But my guess is that the graphics drivers merely let the normal loader do its job and then go and look in the loaded image. So that should work fine.
[x] Definitely interested - but no time to try it manually.
Adding it afterwards would be the best option IMHO, maybe a little tool could just do that to crinkled binaries (with options like +AMD +NV so you don't need to add the AMD string, just in case you already know that you screwed up compatibility with AMD cards).
Would allow you to first do some really expensive compression stuff and decide then, as you proposed.
Adding it afterwards would be the best option IMHO, maybe a little tool could just do that to crinkled binaries (with options like +AMD +NV so you don't need to add the AMD string, just in case you already know that you screwed up compatibility with AMD cards).
Would allow you to first do some really expensive compression stuff and decide then, as you proposed.
And btw. do you intend to add some more compression options for 8k?
Maybe it's possible to have a slightly larger decompressor stub in favor of (even) better compression? That would be something!
Maybe it's possible to have a slightly larger decompressor stub in favor of (even) better compression? That would be something!
Currently crinkling like a god. Just have to leave a "fucking amazing" here...
Z80 version needed :)
Happy anniversary to all Crinkler users!
Ten years ago today, the very first version of Crinkler was released.
We are celebrating this milestone by preparing a major new release. It is not quite ready yet, but we hope to have it out before the end of the week.
As a teaser, here are some feature highlights:
- New header and import code specialized for 1k intros, based on the 1k packer used in several TBC and Loonies 1k intros, but improved and generalized. Typically wins more than 100 bytes on a 1k compared to usual Crinkler compression.
- Ability to export symbols, as suggested by las. This can be used to automatically enable the high-performance GPU on multi-GPU laptops.
- Option to clamp context counters to better compress large, repetitive regions. Usually not a win for 4k, but useful for 8k (suggested by Seven).
- Define a fallback DLL in case a DLL fails to load. For instance, use d3dcompiler_47 if available, otherwise use d3dcompiler_43.
Spread the word to all 1k coders! Assembly is approaching, and we hope the new features will help raise the quality of the 1k compo yet another notch.
Ten years ago today, the very first version of Crinkler was released.
We are celebrating this milestone by preparing a major new release. It is not quite ready yet, but we hope to have it out before the end of the week.
As a teaser, here are some feature highlights:
- New header and import code specialized for 1k intros, based on the 1k packer used in several TBC and Loonies 1k intros, but improved and generalized. Typically wins more than 100 bytes on a 1k compared to usual Crinkler compression.
- Ability to export symbols, as suggested by las. This can be used to automatically enable the high-performance GPU on multi-GPU laptops.
- Option to clamp context counters to better compress large, repetitive regions. Usually not a win for 4k, but useful for 8k (suggested by Seven).
- Define a fallback DLL in case a DLL fails to load. For instance, use d3dcompiler_47 if available, otherwise use d3dcompiler_43.
Spread the word to all 1k coders! Assembly is approaching, and we hope the new features will help raise the quality of the 1k compo yet another notch.
Well holy hell, great job! Get it out before Assembly and I'll be a happy scener!
Happy birthday Crinkler!
Quote:
- Define a fallback DLL in case a DLL fails to load. For instance, use d3dcompiler_47 if available, otherwise use d3dcompiler_43.
Awesome!
So, getting close to the same ballpark as Laturi on the Mac? ;) Happy birthday.
Bump. Not too many days left before compo deadline, I wouldn't mind having the 1k version...
AWESOME.
Would really love to test the Optimus stuff (I know the optimus driver does the right thing on my notebook and I can trivially check whether it works or not).
Would really love to test the Optimus stuff (I know the optimus driver does the right thing on my notebook and I can trivially check whether it works or not).
Getting there. Aiming for a release tonight.
We have the Optimus testing covered, but a test on the corresponding AMD setup would be nice. Anyone here with an AMD dual-GPU laptop?
We have the Optimus testing covered, but a test on the corresponding AMD setup would be nice. Anyone here with an AMD dual-GPU laptop?
Fashionably late... :)
Now a combined 4k and 1k cruncher, Crinkler 2.0 is born!
- New decompression header for 1k intros: smaller, but with a slightly worse compression ratio. The decompressor in this one has quadratic execution time, so don't use it for large (i.e. 4k) intros unless you want to spend minutes on decrunching. Activate with /TINYHEADER.
- New import code for 1k intros: imports all functions from used DLLs into a small hash table and finds a suitable hash function that avoids unwanted collisions. Smaller, but potentially somewhat less robust to future additions of functions to affected DLLs. Activate with /TINYIMPORT.
- Export symbols, either existing symbols or automatically created 32-bit integers, using the /EXPORT option. Can also be used with recompression to add exports to existing intros.
- Do you have big areas of very redundant data? Then the /SATURATE option might be for you.
- Use the /FALLBACKDLL option to define a fallback DLL in case a DLL fails to load. Can be used together with /REPLACEDLL to try a different version first than the one your SDK links to by default.
- Set the alignments of all code sections to 1 using the /UNALIGNCODE option.
- You can now replace DLLs during recompression using the /REPLACEDLL option, provided the DLL names have the same length.
- Header size reduced by 2 bytes. Optimization spotted by qkumba.
- Print previous size of output file for easy comparison.
- Accept values like /SUBSYSTEM:WINDOWS,5.01 which arise from selecting XP compatibility mode in VS.
- Switched from the Intel compiler and OpenMP to the VS 2013 compiler and the MS concurrency API. This is mostly for our own convenience, but also results in a smaller crinkler.exe. The speed seems to be about the same. Unfortunately, this seems to have broken WinE compatibility. Anyone depending on running Crinkler through WinE?
Furthermore, we have internally introduced a new, collision-free hashing method for model estimation and reordering. This gives consistent sizes between model estimation and reordering (the size could jump in weird ways before). It also results in slightly better compression on average, but with large variations, mainly due to the instability of the pseudo-random section reordering. Thus, some intros might turn out slightly bigger than before out of the box. Some tweaking of the contents should be sufficient to bring it below the old size on most cases. We have some ideas for a more robust way to do reordering, but due to the urgency of getting this release out, we are postponing this feature to a later version.
And NOW go and make your 1k! :-D
Now a combined 4k and 1k cruncher, Crinkler 2.0 is born!
- New decompression header for 1k intros: smaller, but with a slightly worse compression ratio. The decompressor in this one has quadratic execution time, so don't use it for large (i.e. 4k) intros unless you want to spend minutes on decrunching. Activate with /TINYHEADER.
- New import code for 1k intros: imports all functions from used DLLs into a small hash table and finds a suitable hash function that avoids unwanted collisions. Smaller, but potentially somewhat less robust to future additions of functions to affected DLLs. Activate with /TINYIMPORT.
- Export symbols, either existing symbols or automatically created 32-bit integers, using the /EXPORT option. Can also be used with recompression to add exports to existing intros.
- Do you have big areas of very redundant data? Then the /SATURATE option might be for you.
- Use the /FALLBACKDLL option to define a fallback DLL in case a DLL fails to load. Can be used together with /REPLACEDLL to try a different version first than the one your SDK links to by default.
- Set the alignments of all code sections to 1 using the /UNALIGNCODE option.
- You can now replace DLLs during recompression using the /REPLACEDLL option, provided the DLL names have the same length.
- Header size reduced by 2 bytes. Optimization spotted by qkumba.
- Print previous size of output file for easy comparison.
- Accept values like /SUBSYSTEM:WINDOWS,5.01 which arise from selecting XP compatibility mode in VS.
- Switched from the Intel compiler and OpenMP to the VS 2013 compiler and the MS concurrency API. This is mostly for our own convenience, but also results in a smaller crinkler.exe. The speed seems to be about the same. Unfortunately, this seems to have broken WinE compatibility. Anyone depending on running Crinkler through WinE?
Furthermore, we have internally introduced a new, collision-free hashing method for model estimation and reordering. This gives consistent sizes between model estimation and reordering (the size could jump in weird ways before). It also results in slightly better compression on average, but with large variations, mainly due to the instability of the pseudo-random section reordering. Thus, some intros might turn out slightly bigger than before out of the box. Some tweaking of the contents should be sufficient to bring it below the old size on most cases. We have some ideas for a more robust way to do reordering, but due to the urgency of getting this release out, we are postponing this feature to a later version.
And NOW go and make your 1k! :-D
After all these years, this release has reminded me just how much there is left for us to do.
I hope to see some 1k intros at assembly putting these ~100bytes to good use :)
I hope to see some 1k intros at assembly putting these ~100bytes to good use :)
what about MASM ? Does Crinkler accept .objs ?
I am actually sitting here somewhat in shock. I recompressed my 1k intro Molten Core and the size went down to 897 bytes...
I thought I would never bother with 1k again since there was no more room to do anything else but now I have enough bytes to maybe do something more interesting.
If I could thumbup again I would.
I thought I would never bother with 1k again since there was no more room to do anything else but now I have enough bytes to maybe do something more interesting.
If I could thumbup again I would.
Goblinish yes. Many 1k windows intros are using asm and crinkler together. I personally used MASM in VS2013 and it worked with crinkler just as seamlessly as coding in C/C++.
@drift: how it looks? I never used VS2013.
Thanks for version 2. Work-in-progress compoentry went from 1138 bytes to 1013.
drift: let me thumbs up for you
goblinish: when you download my 1k it has the source code in it and I also included a complete VS project. It should work on any VS from 2013 or newer which you can download a free version from microsoft. You can see how I set it all up but there is no real tricks, just code asm right in the VS text editor and build it just like any normal C/C++ code. If you have more questions just ask in the 1k code thread on pouet bbs, myself or many other people will be happy to help.
sensenstahl: thankyou!
sensenstahl: thankyou!
g0blinish: Crinkler is not specific to any language, compiler or assembler. As long as it can generate a COFF obj it should work just fine with Crinkler.
Also, this might be a good time to remind everybody that you can still support the Crinkler project by donating intro objs to us for use in our internal test suite. In particular, we seem to be very low on 1k intros :)
Also, this might be a good time to remind everybody that you can still support the Crinkler project by donating intro objs to us for use in our internal test suite. In particular, we seem to be very low on 1k intros :)
@mentor: anyway is better to instal Masm32 and use one than VS2013.
some people said they are used Masm32.
@drift: thanks for the help, will try.
some people said they are used Masm32.
@drift: thanks for the help, will try.
g0blinish: I'm not sure if you are implying that you need Visual Studio for Crinkler. The Visual Studio integration is just a convenience feature. You can also just use Crinkler as a command-line tool.
Thumb for this stuff is cool, and for the prods it may result in!
Thanks for fueling my love/hate for your work!
100+ bytes gain at 1kb is HUGE and I already fell that native 1kb intros had something like a ~1-150bytes edge other web 1kb intros, not to mention the difference of APIs and data ready to use.
100+ bytes gain at 1kb is HUGE and I already fell that native 1kb intros had something like a ~1-150bytes edge other web 1kb intros, not to mention the difference of APIs and data ready to use.
Is it working with MSDOS 4K ? Could it be possible ?
@mentor, I don't get your point, but I think simple code may compact to 1K, e.g.
http://www.pouet.net/prod.php?which=65913
http://www.pouet.net/prod.php?which=65913
g0blinish: just use MASM to generate the obj file and use crinkler to link it? That's exactly what crinkler is for.
blueberry & mentor: nice work!
blueberry & mentor: nice work!
Heeej,
thanx for thie wonderful piece of Software !
Iam running in some troble, compiling my VS2013 project, i followed the Manual, but there are still linking errors...
thanx for thie wonderful piece of Software !
Iam running in some troble, compiling my VS2013 project, i followed the Manual, but there are still linking errors...
Code:
Fehler 4 error LNK: Cannot find symbol '__imp__timeGetTime@0' D:\Programme\DirectXTemplate\MAIN.OBJ DirectXTemplate
Fehler 5 error LNK: Cannot find symbol '___CxxFrameHandler3' D:\Programme\DirectXTemplate\DIRECTXTEMPLATEPCH.OBJ DirectXTemplate
Fehler 6 error LNK: Cannot find symbol '__imp__timeGetTime@0' D:\Programme\DirectXTemplate\MAIN.OBJ DirectXTemplate
Fehler 7 error LNK: Cannot find symbol '__imp_??0id@locale@std@@QAE@I@Z' D:\Programme\DirectXTemplate\DIRECTXTEMPLATEPCH.OBJ DirectXTemplate
Fehler 8 error LNK: Cannot find symbol '__imp_??0id@locale@std@@QAE@I@Z' D:\Programme\DirectXTemplate\DIRECTXTEMPLATEPCH.OBJ DirectXTemplate
Fehler 9 error LNK: Cannot find symbol '__imp_??1exception@std@@UAE@XZ'. D:\Programme\DirectXTemplate\MAIN.OBJ DirectXTemplate
Fehler 10 error LNK: Cannot find symbol '__imp_?_Xbad_alloc@std@@YAXXZ' D:\Programme\DirectXTemplate\DIRECTXTEMPLATEPCH.OBJ DirectXTemplate
Fehler 11 error LNK: Cannot find symbol '__imp_?_Syserror_map@std@@YAPBDH@Z' D:\Programme\DirectXTemplate\DIRECTXTEMPLATEPCH.OBJ DirectXTemplate
Fehler 12 error LNK: Cannot find symbol '__imp_?_Winerror_map@std@@YAPBDH@Z' D:\Programme\DirectXTemplate\DIRECTXTEMPLATEPCH.OBJ DirectXTemplate
Fehler 13 error LNK: Cannot find symbol '__imp_?_Syserror_map@std@@YAPBDH@Z' D:\Programme\DirectXTemplate\DIRECTXTEMPLATEPCH.OBJ DirectXTemplate
Fehler 14 error LNK: Cannot find symbol '__imp_?_Xout_of_range@std@@YAXPBD@Z' D:\Programme\DirectXTemplate\DIRECTXTEMPLATEPCH.OBJ DirectXTemplate
Fehler 15 error LNK: Cannot find symbol '___CxxFrameHandler3' D:\Programme\DirectXTemplate\MAIN.OBJ DirectXTemplate
Fehler 16 error LNK: Cannot find symbol '__imp_?_Xlength_error@std@@YAXPBD@Z' D:\Programme\DirectXTemplate\DIRECTXTEMPLATEPCH.OBJ DirectXTemplate
Fehler 17 error LNK: Cannot find symbol '__imp_?_Xout_of_range@std@@YAXPBD@Z' D:\Programme\DirectXTemplate\DIRECTXTEMPLATEPCH.OBJ DirectXTemplate
Fehler 18 error LNK: Cannot find symbol '__imp___wassert' D:\Programme\DirectXTemplate\MAIN.OBJ DirectXTemplate
Fehler 19 error LNK: Cannot find symbol '__imp___wassert' D:\Programme\DirectXTemplate\MAIN.OBJ DirectXTemplate
Fehler 20 error LNK: Cannot find symbol '__imp_?_Xout_of_range@std@@YAXPBD@Z' D:\Programme\DirectXTemplate\DIRECTXTEMPLATEPCH.OBJ DirectXTemplate
Awesome
Hi nebulus,
First of all, eliminate all uses of the C++ standard library and all uses of exceptions. Disable exceptions in the settings as well.
Then, if you cannot do without some C standard library functions (and thus cannot avoid a standard runtime library), try to link against the msvcrt.lib from VS6. Read the section on standard runtime libraries in the manual for inspiration.
First of all, eliminate all uses of the C++ standard library and all uses of exceptions. Disable exceptions in the settings as well.
Then, if you cannot do without some C standard library functions (and thus cannot avoid a standard runtime library), try to link against the msvcrt.lib from VS6. Read the section on standard runtime libraries in the manual for inspiration.
Quote:
Is it working with MSDOS 4K ? Could it be possible ?
For MSDOS 4ks you would probably want to use a COM file, since these files have no executable header overhead.
Thus, all of the header and import trickery that Crinkler does will not be relevant for MSDOS. A somewhat similar compression algorithm might be useful, but you would probably want it to be 16 bit code and also use much less memory.
Last time I looked, aPACK was one of the most popular packers for MSDOS.
@las: I start the thread, still no result
nebulus:
get rid of your DirectX-Template! Code something on your own by following DirectX-Tutorials to be found everywhere across the web...try to find sth official! ;) (just dont kill everything on exit as Tutorials imply, Windows will do so anyway for you!)
timeGetTime -> i got rid of it by using 4klang-softSynthesizer and using its "time", even providing you a possibility to sync music with your effects! There are other ways to avoid using it for sure, tho! ;)
The rest of your Errors seem to be soluted by just avoiding the standard-runtime aswell. (not too sure about this, because of i have no idea what this DirectX-Template is you´re using!)
Another problem you might run into is not being able to use "math.h"...
...first of all: try to avoid as much C-code as possible as it won´t pack away as good as your shader-code in Ascii! (have a large array of Ascii-Shader-Code to be compiled at runtime into shadercode, directX has a runtime-shadercompiler, use it!)
...secondary: if you cannot avoid using some math in your C-code, use the CPU directly via Assembler-Intrinsics, the only Assembler-Code you´ll ever need, except you want to squeeze out some more very few bytes lateron (never went that far myself!):
Sorry, Pouet still seems to mess up(->not recognize!) TABULATORS when pasting Code! :/
get rid of your DirectX-Template! Code something on your own by following DirectX-Tutorials to be found everywhere across the web...try to find sth official! ;) (just dont kill everything on exit as Tutorials imply, Windows will do so anyway for you!)
timeGetTime -> i got rid of it by using 4klang-softSynthesizer and using its "time", even providing you a possibility to sync music with your effects! There are other ways to avoid using it for sure, tho! ;)
The rest of your Errors seem to be soluted by just avoiding the standard-runtime aswell. (not too sure about this, because of i have no idea what this DirectX-Template is you´re using!)
Another problem you might run into is not being able to use "math.h"...
...first of all: try to avoid as much C-code as possible as it won´t pack away as good as your shader-code in Ascii! (have a large array of Ascii-Shader-Code to be compiled at runtime into shadercode, directX has a runtime-shadercompiler, use it!)
...secondary: if you cannot avoid using some math in your C-code, use the CPU directly via Assembler-Intrinsics, the only Assembler-Code you´ll ever need, except you want to squeeze out some more very few bytes lateron (never went that far myself!):
Code:
//___sInUs___
#pragma code_seg(".crtemuf")
float ASMsinf(float i)
{ __asm fld i
__asm fsin
}
//___cOsInUs___
#pragma code_seg(".crtemuf")
float ASMcosf(float i)
{ __asm fld i
__asm fcos
}
//___rOUnd_flOAt_tO_IntEgEr___
#pragma code_seg(".crtemuf")
int ASMlrintf (float flt)
{ int reti;
__asm
{
fld flt
fistp reti // rounds ;)
}
return reti;
}
//___sqUArE_rOOt___
#pragma code_seg(".crtemuf")
float ASMsqrtf(float i)
{ __asm fld i
__asm fsqrt
}
//___mOdUlO_flOAt___
#pragma code_seg(".crtemuf")
float ASMfmodf(float i, float j)
{ __asm fld j
__asm fld i
__asm fprem
__asm fxch
__asm fstp i
}
//___AbsOlUtE_flOAt___
#pragma code_seg(".crtemuf")
float ASMfabsf(float i)
{ __asm fld i
__asm fabs
}
//___clEAr_mEmOry___
#pragma code_seg(".crtemui")
void ASMZeroMemory(void* dest, SIZE_T s)
{ __asm mov edi, dest
__asm xor eax, eax
__asm mov ecx, s
__asm rep stosb
}
//___cOpy_mEmOry___
#pragma code_seg(".crtemui")
void ASMCopyMemory(void* dest, void* souAe, SIZE_T s)
{ __asm mov esi, souAe
__asm mov edi, dest
__asm mov ecx, s
__asm rep movsb
}
//___rAndOm_nUmbEr_gEnErAtOr___
#pragma data_seg(".rand")
DWORD RandSeed;
#pragma code_seg(".crtemui")
unsigned long random()
{
RandSeed = (RandSeed * 196314165) + 907633515;
return RandSeed;
}
Sorry, Pouet still seems to mess up(->not recognize!) TABULATORS when pasting Code! :/
pouet seems to have added some whitespace aswell:
ASMCopyMemory()
ASMZeroMemory()
Just remove the Whitespace after "ASM" ;)
ASMCopyMemory()
ASMZeroMemory()
Just remove the Whitespace after "ASM" ;)
on top of your main-class, NOT the main()-function:
may help aswell!
You may have recognized those "#pragma"s everywhere, crinkler.manual has info for what those are! More than just useful for debugging, also helps crinkler itself to recognize how to handle data/code and crunch it for you! ;) Use it everywhere, as often as you want, even repeat names for them, if you think you have sth that could be the same sort of data/code...experimenting is one thing you can try if you run out of ideas getting it smaller, just some few bytes away from the final bytesize...but try planning without having to do so once you got a hang to it! ;)
Code:
#define WIN32_LEAN_AND_MEAN
#define WIN32_EXTRA_LEAN
#pragma code_seg(".fltused")
extern "C" { int _fltused = 1; }
may help aswell!
You may have recognized those "#pragma"s everywhere, crinkler.manual has info for what those are! More than just useful for debugging, also helps crinkler itself to recognize how to handle data/code and crunch it for you! ;) Use it everywhere, as often as you want, even repeat names for them, if you think you have sth that could be the same sort of data/code...experimenting is one thing you can try if you run out of ideas getting it smaller, just some few bytes away from the final bytesize...but try planning without having to do so once you got a hang to it! ;)
Quote:
Another problem you might run into is not being able to use "math.h"...
You can actually get most of this without resorting to inline assembly, simply by enabling intrinsic functions in the VS compiler settings.
I was depending on crinkler working on wine... but I haven't touched PC size coding for quite some time now...
I investigated the Wine issue a bit more. In the newest version from the git repository it actually works. It spits out lots of warnings, but it runs.
This is mainly due to the GetNumaHighestNodeNumber function returning a sensible stub value (always 0, so it will only take advantage of one socket in a multi-socket machine), rather than returning garbage.
This is mainly due to the GetNumaHighestNodeNumber function returning a sensible stub value (always 0, so it will only take advantage of one socket in a multi-socket machine), rather than returning garbage.
Crinkler is working on 64-bit Windows 10, no problem. We are happy.
This tool has changed the 4k landscape for good.
Wine just came out with a new version - 1.8 - so I tested Crinkler 2.0 using this new version. It works great, with the previously mentioned caveat that in a multi-socket system, only the cores in one physical CPU will be used. Shouldn't make much difference for most users I guess...
So now you can again log in to your beefy server to do the crinkling. :)
Be aware that RVA forwarding may be different in Wine than on real Windows. For example, timeGetTime is forwarded to GetTickCount. For this particular case, it will probably work out well, but the other way around - when something is forwarded on Windows but not in Wine - will break.
So now you can again log in to your beefy server to do the crinkling. :)
Be aware that RVA forwarding may be different in Wine than on real Windows. For example, timeGetTime is forwarded to GetTickCount. For this particular case, it will probably work out well, but the other way around - when something is forwarded on Windows but not in Wine - will break.
Thank you for doing this. :)
Crinkler crashes when using gdi32.lib from the Windows 10 Anniversary Update (10.0.14393.0). An older version of the library (10.0.10586.0) works fine. The log looks like this:
The dump file reveals the exception code 0xC0000005.
Code:
Loading kernel32.lib...
Loading user32.lib...
Loading opengl32.lib...
Loading Gdi32.lib...
Oops! Crinkler has crashed.
Dump files written to dump0000_mini.dmp and dump0000_full.dmpThe dump file reveals the exception code 0xC0000005.
I can confirm that bug. Crashes here as well. Using the 8.1 SDK works fine.
Thank you for the bug report MrYeah and xTr1m.
I finally managed to track down the right gdi32.lib file and reproduce the crash. My earlier attempts at installing the SDKs didn't seem to give me those standard lib files for some reason...
We will investigate. :)
I finally managed to track down the right gdi32.lib file and reproduce the crash. My earlier attempts at installing the SDKs didn't seem to give me those standard lib files for some reason...
We will investigate. :)
missing thumb
Hello crinkler dont work with VS2017 now.
BB can you patch Crinkler?
Thanks
BB can you patch Crinkler?
Thanks
Hi Ciclope.
What do you mean it doesn't work? What happens when you do what?
Which SDK version are you using? Could it be the same bug that MrYeah and xTr1m reported?
What do you mean it doesn't work? What happens when you do what?
Which SDK version are you using? Could it be the same bug that MrYeah and xTr1m reported?
ooh, forgot the quite obvious thumb up for this wizardry! Used it a lot. Thank you <3
guess what :)
Hi BB,
is not the same bug like MrYeah and xTr1m. I use on the VS 2017 environment sdk 10.0.14393.0.
VS 2017 break with link.exe was stopt with code -1
If i use the VS 2017 linker all work fine.
On my VS2015 environment the same project work with crinkler.
Ciclope
is not the same bug like MrYeah and xTr1m. I use on the VS 2017 environment sdk 10.0.14393.0.
VS 2017 break with link.exe was stopt with code -1
If i use the VS 2017 linker all work fine.
On my VS2015 environment the same project work with crinkler.
Ciclope
Oh sorry its the same bug !!!!
This is a great tool. If you ever plan to release a crinkler version which is able to handle x64 executables this small blog entry may be helpfull for you: https://drakopensulo.wordpress.com/2017/08/06/smallest-pe-executable-x64-with-every-byte-executed/
Amazing prod, thank you for such great tool!!!
I have some issues with version 2.0, for some reason results packed with it work on my win7 desktop and doesn't work on other systems.
I don't have such problem with version 1.4.
Also the size of the result is different on differeent Windows systems, is it due different versions of libraries or something else?
Also I have issue with Windows 10 VS 2015
it breaks with link.exe was stop with code -1
To solve this issue you have to disable /PROGRESSGUI
It appears in Win10 and no issue with Win7
I have some issues with version 2.0, for some reason results packed with it work on my win7 desktop and doesn't work on other systems.
I don't have such problem with version 1.4.
Also the size of the result is different on differeent Windows systems, is it due different versions of libraries or something else?
Also I have issue with Windows 10 VS 2015
it breaks with link.exe was stop with code -1
To solve this issue you have to disable /PROGRESSGUI
It appears in Win10 and no issue with Win7
Quote:
I have some issues with version 2.0, for some reason results packed with it work on my win7 desktop and doesn't work on other systems.
I don't have such problem with version 1.4.
Also the size of the result is different on differeent Windows systems, is it due different versions of libraries or something else?
Am I right in assuming that you are using the TINYIMPORT option? Behavior like what you describe is sort of expected when using that feature. It builds a minimal hash table which has no collisions between the imported functions and other functions in the same DLL, but if a later Windows version adds a function which collides with an imported function, the intro may break.
To minimize the risk of breakage, Crinkler knows about all functions in commonly used DLLs in all Windows versions at the time of its release and avoids these in addition to the ones present on the build machine, but since Crinkler 2.0 is now more than two years old, some disrupting functions have snuck in since. Time to release an update, I guess. :)
There is some more discussion of the feature in this thread.
If you are not using the TINYIMPORT option, then you have found some really unexpected behavior. In any case, if you could send us your object files, the options you use when compressing, and the resulting executable when compressing on your machine, that would be very helpful.
Quote:
Also I have issue with Windows 10 VS 2015
it breaks with link.exe was stop with code -1
To solve this issue you have to disable /PROGRESSGUI
It appears in Win10 and no issue with Win7
Interesting. We will take a look at why that could be, though it could be hard to reproduce.
Thank you for the bug report!
1. Yeah you right.. I use TINYIMPORT some times, but I need to retest to make sure!
can I track that somehow?
2.
Wow ;)
3.
thank you ;)!!! I'll move there
4.
I will send you all the project, with the code ;) np.. Ijust need some time to prepare everything!
BTW it would be great to have some feature to use default linker, I have a lot of builds for one project and I am trying to automate the building process and as result I want to have a folder with all binary types..
for instance:
4kb slow 1920x1080
4kb slow 1280x720
4kb slow 1680x1050
4kb slow 1024x768
fast build 1920x1080
CAPTURE BUILD
4klang CAPTURE only build
normal builds to check on different machines
a slow build takes about 2-5 minutes.. thus I want to run Batch Build and make a coffee... ;)
Quote:
It builds a minimal hash table which has no collisions between the imported functions and other functions in the same DLL, but if a later Windows version adds a function which collides with an imported function, the intro may break.
can I track that somehow?
2.
Quote:
but since Crinkler 2.0 is now more than two years old, some disrupting functions have snuck in since. Time to release an update, I guess. :)
Wow ;)
3.
Quote:
There is some more discussion of the feature in this thread.
thank you ;)!!! I'll move there
4.
Quote:
In any case, if you could send us your object files, the options you use when compressing, and the resulting executable when compressing on your machine, that would be very helpful.
I will send you all the project, with the code ;) np.. Ijust need some time to prepare everything!
BTW it would be great to have some feature to use default linker, I have a lot of builds for one project and I am trying to automate the building process and as result I want to have a folder with all binary types..
for instance:
4kb slow 1920x1080
4kb slow 1280x720
4kb slow 1680x1050
4kb slow 1024x768
fast build 1920x1080
CAPTURE BUILD
4klang CAPTURE only build
normal builds to check on different machines
a slow build takes about 2-5 minutes.. thus I want to run Batch Build and make a coffee... ;)
Hello!
I can't run any intros packed by Crinkler (on Win 10 x64 / Core i5-2500K). But Win 7 x86 on VMware can run some of them. And my notebook with Win 10 x64 (Core i5 too) can run some intros but most of them glitches.
And the main problem is that I can't run anything that I linked by Crinkler.
Simply "Hello World" program hangs with core load (it seems like it unpacks forever). Or does AV exception if I link with /hashsize:1 option (~ after 10 secs).
Please look this: https://cloud.mail.ru/public/EeVT/qXJHCEBfp
I can't run it successfully anywhere: neither on my PC, nor on notebook nor on VMware with Win7 on XP.
p.s. This archive contains ASSEMBLER sources (fasm, nasm, masm, uasm) and batch files for compilation. And binaries also.
What's wrong???
I can't run any intros packed by Crinkler (on Win 10 x64 / Core i5-2500K). But Win 7 x86 on VMware can run some of them. And my notebook with Win 10 x64 (Core i5 too) can run some intros but most of them glitches.
And the main problem is that I can't run anything that I linked by Crinkler.
Simply "Hello World" program hangs with core load (it seems like it unpacks forever). Or does AV exception if I link with /hashsize:1 option (~ after 10 secs).
Please look this: https://cloud.mail.ru/public/EeVT/qXJHCEBfp
I can't run it successfully anywhere: neither on my PC, nor on notebook nor on VMware with Win7 on XP.
p.s. This archive contains ASSEMBLER sources (fasm, nasm, masm, uasm) and batch files for compilation. And binaries also.
What's wrong???
Hi jin_x
It sounds like your antivirus is interfering. Can you run the Crinkler-compressed intros if you disable your AV or whitelist the directory containing the exes?
It sounds like your antivirus is interfering. Can you run the Crinkler-compressed intros if you disable your AV or whitelist the directory containing the exes?
Hi Blueberry.
I tried to run on different systems (including from VMware, USB WinPE, etc) with switched off (and absent) antivirus and DEP, run as administrator etc... The result is the same.
Can you download my archive from the link above and try (run and compile) yourself?
It contains compiled (and linked by Crinkler) and sources. There's nothing hard to understand...
When I link it with '/hashsize:1' option it crashes after ~ 10 secs. Under OllyDbg I see that it crashes trying to 'stosb' to edi=0053C000 (esp=01C3FF84).
You can look screenshots of crash on XP (VMware) here: https://cloud.mail.ru/public/4BKP/xGv6TuRnh
(and I doubled my archive 'crinkler_test.zip' here too).
Moreover I can't run ANY linked by Crinkler intro on my main system...
I tried to run on different systems (including from VMware, USB WinPE, etc) with switched off (and absent) antivirus and DEP, run as administrator etc... The result is the same.
Can you download my archive from the link above and try (run and compile) yourself?
It contains compiled (and linked by Crinkler) and sources. There's nothing hard to understand...
When I link it with '/hashsize:1' option it crashes after ~ 10 secs. Under OllyDbg I see that it crashes trying to 'stosb' to edi=0053C000 (esp=01C3FF84).
You can look screenshots of crash on XP (VMware) here: https://cloud.mail.ru/public/4BKP/xGv6TuRnh
(and I doubled my archive 'crinkler_test.zip' here too).
Moreover I can't run ANY linked by Crinkler intro on my main system...
The problem was with libs of MASM32.
It will work ok when change libpath to Windows SDK :))
It will work ok when change libpath to Windows SDK :))
Thanks for the pointer. I will take a closer look at the lib files in your archive to see what goes wrong. But how does this affect your ability to run other Crinkler-compressed intros?? Something else must be interfering here.
Hi Pouet :)
Crinkler development has been a bit stagnant lately - something we hope to change soon. Meanwhile, bugs are bugs, and bugs must be fixed. So here's a small bugfix release: Crinkler 2.0a!
- The crashing issue with recent Windows SDK versions reported by several people here has been fixed. As we have seen on earlier occasions, Microsoft sometimes interprets their own specifications somewhat liberally. ;)
- It turns out that the crashes that some people have observed when using functions from ole32.dll (for instance to access the Microsoft Speech API) was caused by the same bug. So if SAPI is your thing, you can now use it again. Just be aware that the forwards have changed between every recent Windows version (7, 8 and 10), so your program will probably only work on the Windows version on which it is built.
- Corrected a horizontal alignment issue in the HTML report. It was a bit jaggy, but it is nice and straight now. :)
- Fixed a couple of issues with /TINYIMPORT: Forwarded RVAs were not supported, and MessageBox was always imported, even if it was not used.
- Updated the internal function list used by /TINYIMPORT to include functions added up to including the Spring Creators Update 2018. This should hopefully increase the compatibility of /TINYIMPORT, at least for a time.
- Crinkler now prints a warning when /TINYIMPORT is used, recommending the inclusion of a compatible version using the normal import mechanism. The section on /TINYIMPORT in the manual has also been extended with a more thorough description of the mechanism and a stronger warning.
- The link in the manual to the lib file for using the VS6 msvcrt.dll had ceased to work. A new, working link has taken its place.
Enjoy! :-D
Crinkler development has been a bit stagnant lately - something we hope to change soon. Meanwhile, bugs are bugs, and bugs must be fixed. So here's a small bugfix release: Crinkler 2.0a!
- The crashing issue with recent Windows SDK versions reported by several people here has been fixed. As we have seen on earlier occasions, Microsoft sometimes interprets their own specifications somewhat liberally. ;)
- It turns out that the crashes that some people have observed when using functions from ole32.dll (for instance to access the Microsoft Speech API) was caused by the same bug. So if SAPI is your thing, you can now use it again. Just be aware that the forwards have changed between every recent Windows version (7, 8 and 10), so your program will probably only work on the Windows version on which it is built.
- Corrected a horizontal alignment issue in the HTML report. It was a bit jaggy, but it is nice and straight now. :)
- Fixed a couple of issues with /TINYIMPORT: Forwarded RVAs were not supported, and MessageBox was always imported, even if it was not used.
- Updated the internal function list used by /TINYIMPORT to include functions added up to including the Spring Creators Update 2018. This should hopefully increase the compatibility of /TINYIMPORT, at least for a time.
- Crinkler now prints a warning when /TINYIMPORT is used, recommending the inclusion of a compatible version using the normal import mechanism. The section on /TINYIMPORT in the manual has also been extended with a more thorough description of the mechanism and a stronger warning.
- The link in the manual to the lib file for using the VS6 msvcrt.dll had ceased to work. A new, working link has taken its place.
Enjoy! :-D
Incredible tool.
I've been having issues with it crashing out under certain conditions - massive props to Mentor for pinpointing it to an odd interaction between the PROGRESSGUI and the DisplayFusion software I have installed. I'm now back in business, but just a heads-up for anyone else running DisplayFusion
I've been having issues with it crashing out under certain conditions - massive props to Mentor for pinpointing it to an odd interaction between the PROGRESSGUI and the DisplayFusion software I have installed. I'm now back in business, but just a heads-up for anyone else running DisplayFusion
No bugs, no crashing.. Just (in time for) Revision
The link.exe trick doesn't seem to work in Visual Studio 2017 anymore. It didn't work on my previous Win7 installation, and not on Win10 either. Crinkler is just ignored entirely without getting invoked and VS uses the default linker. Anyone else had this? Any solutions?
Didn't take too long (thanks msqrt). Just put "$(SolutionDir)" under your VC++ Executable directories in the configuration settings.
Hi,
I would like to request a feature be added to Crinkler. I want to use Crinkler for small, data compression programs.
Here is an example - Please see the cpp, obj, and exe (created with crinkler 2.0a) files in this archive: https://www.dropbox.com/s/qw0drwzqqck26f7/lzss1.7z?dl=0
As you can see, the exe runs ok. But, when we try to send an argument via argv (i.e. to compress a file or decompress a file), the exe ignores the arguments are returns the main printf statement and exits.
If a feature could be added to support this, it would be greatly appreciated.
Thanks!
I would like to request a feature be added to Crinkler. I want to use Crinkler for small, data compression programs.
Here is an example - Please see the cpp, obj, and exe (created with crinkler 2.0a) files in this archive: https://www.dropbox.com/s/qw0drwzqqck26f7/lzss1.7z?dl=0
As you can see, the exe runs ok. But, when we try to send an argument via argv (i.e. to compress a file or decompress a file), the exe ignores the arguments are returns the main printf statement and exits.
If a feature could be added to support this, it would be greatly appreciated.
Thanks!
Hi dnm221
Commandline arguments are handled by the startup code in msvcrt.lib. Since you are already using msvcrt for the various C stdlib functions your code uses, all you have to do to include the startup code is to not specify a custom entry point.
This doesn't quite work though. The startup code also handles the execution of various initializers that need to run before main. It assumes that two lists of function pointers are stored between special labels (one between ___xi_a and ___xi_z and one between ___xc_a and ___xc_z). Crinkler does not treat those labels specially, so they end up in (more or less) random places in the uninitialized section. The startup code tries to call everything between the start and end labels and crashes.
There is a small trick that will sometimes work: add plenty of ORDERTRIES (1000 or more). This is highly likely to place these labels next to each other (since that will make the startup code compress better). Then the code works. You can check their placement in the compression report.
This still pulls in around 200 bytes of code for the startup, much of which is probably not needed. A smaller solution is to use this minimal startup code (nasm/yasm syntax):
Commandline arguments are handled by the startup code in msvcrt.lib. Since you are already using msvcrt for the various C stdlib functions your code uses, all you have to do to include the startup code is to not specify a custom entry point.
This doesn't quite work though. The startup code also handles the execution of various initializers that need to run before main. It assumes that two lists of function pointers are stored between special labels (one between ___xi_a and ___xi_z and one between ___xc_a and ___xc_z). Crinkler does not treat those labels specially, so they end up in (more or less) random places in the uninitialized section. The startup code tries to call everything between the start and end labels and crashes.
There is a small trick that will sometimes work: add plenty of ORDERTRIES (1000 or more). This is highly likely to place these labels next to each other (since that will make the startup code compress better). Then the code works. You can check their placement in the compression report.
This still pulls in around 200 bytes of code for the startup, much of which is probably not needed. A smaller solution is to use this minimal startup code (nasm/yasm syntax):
Code:
extern __imp____getmainargs
extern __imp__ExitProcess@4
extern _main
global _mainCRTStartup
section startup text align=1
_mainCRTStartup:
push eax
mov dword [esp], esp
push eax
mov dword [esp], esp
push eax
mov dword [esp], esp
call [__imp____getmainargs]
call _main
push eax
call [__imp__ExitProcess@4]
Hi experts,
how to link assembler source code that has a resource file?! Does this compression linker not understand the resource files at all? Are there any similar plans for the future to make support resources?
Thanks for the great project!
how to link assembler source code that has a resource file?! Does this compression linker not understand the resource files at all? Are there any similar plans for the future to make support resources?
Thanks for the great project!
Assembling: exmem.asm
Crinkler 2.0a (Mar 28 2018) (c) 2005-2018 Aske Simon Christensen & Rune Stubbe
Target: exmemcr.exe
Tiny compressor: YES
Tiny import: NO
Subsystem type: CONSOLE
Large address aware: NO
Compression mode: SLOW
Saturate counters: NO
Hash size: 100 MB
Hash tries: 20
Order tries: 1000
Report: exmem.html
Transforms: CALLS
Replace DLLs: NONE
Fallback DLLs: NONE
Range DLLs: NONE
Exports: NONE
Loading exmem.obj...
C:\MASM64\EXMEM.OBJ: error LNK: Unsupported file type
Press any key to continue . . .
Crinkler 2.0a (Mar 28 2018) (c) 2005-2018 Aske Simon Christensen & Rune Stubbe
Target: exmemcr.exe
Tiny compressor: YES
Tiny import: NO
Subsystem type: CONSOLE
Large address aware: NO
Compression mode: SLOW
Saturate counters: NO
Hash size: 100 MB
Hash tries: 20
Order tries: 1000
Report: exmem.html
Transforms: CALLS
Replace DLLs: NONE
Fallback DLLs: NONE
Range DLLs: NONE
Exports: NONE
Loading exmem.obj...
C:\MASM64\EXMEM.OBJ: error LNK: Unsupported file type
Press any key to continue . . .
Hi Zerochan
Crinkler does not support resources. We have not encountered a use case for it that seemed relevant for small intros. What do you want to use it for?
The Resource Table RVA in the Crinkler header currently overlaps with some of the decompression code. So resource support could be somewhat cumbersome, as it involves moving some code around, especially if we want to do it in a way which does not have any size overhead when resources are not used. But I guess it could be done. (The situation is different for exports, as the Export Table RVA has to be valid in any case, even if there are no exports.)
Crinkler does not support resources. We have not encountered a use case for it that seemed relevant for small intros. What do you want to use it for?
The Resource Table RVA in the Crinkler header currently overlaps with some of the decompression code. So resource support could be somewhat cumbersome, as it involves moving some code around, especially if we want to do it in a way which does not have any size overhead when resources are not used. But I guess it could be done. (The situation is different for exports, as the Export Table RVA has to be valid in any case, even if there are no exports.)
Crinkler 2.1 is available, featuring massive speed optimizations!
The Crinkler executable is now available in both 32 and 64 bit versions. Crinkler still only produces 32 bit executables.
The main new feature in this release is the /REUSE option. It allows Crinkler to reuse the models, section ordering and hash table size from a previous run. When this option is used, Crinkler writes a text file containing this information at the end of compression and reads it back in at the beginning. This has two main advantages:
- The compression speed is similar to INSTANT compression, but since the models and ordering is tuned to your intro, the ratio mostly carries over from the previous run.
- The compression is much more stable against small changes in the input, allowing you to quickly iterate on your content and get a reliable estimation of the actual size impact of your changes.
Big thanks to Seven for beta testing this feature.
Did I say speed optimizations? Crinkler 2.1 features:
- A new, vectorized model estimator, giving a speedup of typically 8-12x. The resulting models can be different from the ones that 2.0 find, but the size seems to be slightly better on average.
- Optimized internal hash function for section reordering, resulting in a 3-4x speedup. The behavior of section reordering is exactly the same as before, but since the model estimation is different, you might still end up in a very different place (sometimes better, sometimes worse).
- Optimized and parallellized hash table size optimization. Speedup dependent on number of cores. The default number of HASHTRIES have been increased to 100. Also the default HASHSIZE has been increased to 500, which is more in line with what is reasonable on today's hardware.
For those of you who absolutely want Crinkler to be slow, we have added a new VERYSLOW compression mode, which is about 5-10x as slow as SLOW and sometimes improves the size by a few bytes. The default compression mode is now SLOW, since it actually isn't so slow anymore. :)
The HTML compression report has received a slight facelift:
- The report now includes the size of the output file.
- The bits-per-byte color legend is more compact, so it doesn't waste so much space at the top.
- The links to collapse/expand/show/hide various parts of the report have been replaced by a set of configurations of what is shown/expanded.
- The code columns have been adjusted to give more space to the opcode and operand columns.
- The data hex view has been expanded from 16 to 32 columns.
Finally, Crinkler would crash when it failed to open an existing input file (due to permission problems, for instance). It now aborts with an error message. Thanks to noby for reporting the bug.
Have a merry Christmas everybody! :-D
The Crinkler executable is now available in both 32 and 64 bit versions. Crinkler still only produces 32 bit executables.
The main new feature in this release is the /REUSE option. It allows Crinkler to reuse the models, section ordering and hash table size from a previous run. When this option is used, Crinkler writes a text file containing this information at the end of compression and reads it back in at the beginning. This has two main advantages:
- The compression speed is similar to INSTANT compression, but since the models and ordering is tuned to your intro, the ratio mostly carries over from the previous run.
- The compression is much more stable against small changes in the input, allowing you to quickly iterate on your content and get a reliable estimation of the actual size impact of your changes.
Big thanks to Seven for beta testing this feature.
Did I say speed optimizations? Crinkler 2.1 features:
- A new, vectorized model estimator, giving a speedup of typically 8-12x. The resulting models can be different from the ones that 2.0 find, but the size seems to be slightly better on average.
- Optimized internal hash function for section reordering, resulting in a 3-4x speedup. The behavior of section reordering is exactly the same as before, but since the model estimation is different, you might still end up in a very different place (sometimes better, sometimes worse).
- Optimized and parallellized hash table size optimization. Speedup dependent on number of cores. The default number of HASHTRIES have been increased to 100. Also the default HASHSIZE has been increased to 500, which is more in line with what is reasonable on today's hardware.
For those of you who absolutely want Crinkler to be slow, we have added a new VERYSLOW compression mode, which is about 5-10x as slow as SLOW and sometimes improves the size by a few bytes. The default compression mode is now SLOW, since it actually isn't so slow anymore. :)
The HTML compression report has received a slight facelift:
- The report now includes the size of the output file.
- The bits-per-byte color legend is more compact, so it doesn't waste so much space at the top.
- The links to collapse/expand/show/hide various parts of the report have been replaced by a set of configurations of what is shown/expanded.
- The code columns have been adjusted to give more space to the opcode and operand columns.
- The data hex view has been expanded from 16 to 32 columns.
Finally, Crinkler would crash when it failed to open an existing input file (due to permission problems, for instance). It now aborts with an error message. Thanks to noby for reporting the bug.
Have a merry Christmas everybody! :-D
thank you blueberry!!
and mentor :)
the best christmas present, thank you!
I tried it latest version with my 4k intros and I replicate any speed improvements? All run times are within a second of the previous 2.0a version while compressing to the same result. This was using both versions with the same invocation command, or is there something additional I need to do to speed it up? (fwiw I don't think this was really an issue for me, even with heavy settings a typical crinking operation for final release takes only about a minute.)
The /VERYSLOW option seems promising though, managing to squeeze out a few more bytes consistently =)
The /VERYSLOW option seems promising though, managing to squeeze out a few more bytes consistently =)
and i cannot* replicate...
Ah, nevermind... trying now a day later there def is a difference, going down from about 1m20s to 28 seconds using the same settings =)
It's almost a tradition by now to have a small bugfix release after each major release. ;)
Crinkler 2.1a is up. Just a few, small fixes this time:
- It turned out the compression report wasn't wide enough for the new 32-column hex view at some zoom levels (such as 100%). This made the report widen locally when unfolding a data section in an aesthetically displeasing way. The report has been made slightly wider overall to fix this.
- New /REUSEMODE:WRITE option, which will cause Crinkler to write the reuse file without reading it in. This can be used when running without reuse in order to avoid having to do another run when switching reuse on.
Crinkler 2.1a is up. Just a few, small fixes this time:
- It turned out the compression report wasn't wide enough for the new 32-column hex view at some zoom levels (such as 100%). This made the report widen locally when unfolding a data section in an aesthetically displeasing way. The report has been made slightly wider overall to fix this.
- New /REUSEMODE:WRITE option, which will cause Crinkler to write the reuse file without reading it in. This can be used when running without reuse in order to avoid having to do another run when switching reuse on.
Is it possible to increase maximum allowed out.exe size from 128K to 256K or 512K? My use-case for Crinkler these days is slightly atypical and it would be very helpful for me if this feature could be added.
It is possible in principle. The 128k limitation comes from the fact that the address where the (decompressed) code and data are located is hardcoded to 0x420000, which is 128k after the ImageBase (executable load address) of 0x400000. Lifting the limitation is thus not just a matter of changing a constant somewhere, but will involve adding an option to move the code base address. This is something we have considered doing.
Have you tried kkrunchy on your executable? I would expect it to beat the compression ratio of Crinkler at those sizes. Or is there a reason that you want to use Crinkler specifically?
Have you tried kkrunchy on your executable? I would expect it to beat the compression ratio of Crinkler at those sizes. Or is there a reason that you want to use Crinkler specifically?
Quote:
It is possible in principle. The 128k limitation comes from the fact that the address where the (decompressed) code and data are located is hardcoded to 0x420000, which is 128k after the ImageBase (executable load address) of 0x400000. Lifting the limitation is thus not just a matter of changing a constant somewhere, but will involve adding an option to move the code base address. This is something we have considered doing.
Have you tried kkrunchy on your executable? I would expect it to beat the compression ratio of Crinkler at those sizes. Or is there a reason that you want to use Crinkler specifically?
Crinkler is superior to any exe packer because it does not produce nearly as many A/V false flags, which is quite important in my case.
Any chance for a switch which denies stripping unused code (/OPT:NOREF equivalent)?
I'd like to mess with ImportTable by hardcoded offsets (it's a 3 byte call instead of 6 bytes), but this makes declared but indirectly called functions to be stripped away from ImportTable...
Great work anyway.
I'd like to mess with ImportTable by hardcoded offsets (it's a 3 byte call instead of 6 bytes), but this makes declared but indirectly called functions to be stripped away from ImportTable...
Great work anyway.
Quote:
Any chance for a switch which denies stripping unused code (/OPT:NOREF equivalent)?
That would bring in lots of unused parts, especially from the lib files. And it would still only include the imports that are actually referenced in some of the code. This doesn't sound like it is what you want.
Quote:
I'd like to mess with ImportTable by hardcoded offsets (it's a 3 byte call instead of 6 bytes), but this makes declared but indirectly called functions to be stripped away from ImportTable...
So you want a way to explicitly include a function in the import table even though nothing refers to it, and then somehow access the index at which it is placed, so you can use it in your code. Is this correct?
I have actually thought about adding a feature like this, since I will probably need it for one of my own projects (to index dynamically into the import table based on a bytecode value). No plans for when it will be implemented, though. :)
i have never used crinkler and maybe never will, but given all the demoscene pearlz i enjoyed, made possible with crinkler, up is the only direction a thumb can have here =)
One of the 4k scene foundation stones together with 4klang (and shadertoy)
Quote:
So you want a way to explicitly include a function in the import table even though nothing refers to it, and then somehow access the index at which it is placed, so you can use it in your code. Is this correct?
Almost exactly ;) The true sentence is: even though nothing >>directly<< refers to it.
I know the index in ImportTable as long, as the function is included. I just need to peek it up in Crinkler report and hardcode it back in my code.
To be strict, instead of:
Code:
EXTERN _imp__ShowWindow@4:PROC
...
call DWORD PTR [_imp_ShowWindow@4]; 2 + 4 bytes
use:
Code:
EXTERN ImportTable
lea EDI,ImportTable
...
call DWORD PTR [EDI + offset to _imp_ShowWindow@4]; 2 + 1 bytes
Right now the latter is only possible if the function was directly referenced before (but for a 1K majority of functions is called only once).
It is unlikely that the indexed version will be any smaller after compression, actually. Crinkler quickly learns to recognize the
Also, this trick will not work for 1k (that is, when using /TINYIMPORT), since in that case the imported function pointers are spread out over a larger area, so a single byte is not enough to index them. In a sense, /TINYIMPORT stores more information (the function hash) in each call instead of including a 4 byte hash (not shown in the report) for each imported function.
Code:
pattern, so after just a few of such calls, they compress to less than 2 bytes each. The indexed ones will not be significantly smaller, since there is just as much entropy in them.call dword [function]Also, this trick will not work for 1k (that is, when using /TINYIMPORT), since in that case the imported function pointers are spread out over a larger area, so a single byte is not enough to index them. In a sense, /TINYIMPORT stores more information (the function hash) in each call instead of including a 4 byte hash (not shown in the report) for each imported function.
That's a very acceptable explanation. I have already learned some of the Crinklers specifics and I know that shorter code not always is a better compressed one...
Of course, maybe I missed some side effects, but it seems like they should work from my little tests. Save two bytes in decompress:
instead of ror [esi],cl / rol [esi],cl to get bit, just use bt [esi],ecx
also at 00E5: dec ecx / jnz 007D, loop 007D does the same.
Any chance it will be open sourced? I would like to make a version for 64-bit executables: https://board.flatassembler.net/topic.php?t=21094
instead of ror [esi],cl / rol [esi],cl to get bit, just use bt [esi],ecx
also at 00E5: dec ecx / jnz 007D, loop 007D does the same.
Any chance it will be open sourced? I would like to make a version for 64-bit executables: https://board.flatassembler.net/topic.php?t=21094
Quote:
off by one error, doesn't really work that way.instead of ror [esi],cl / rol [esi],cl to get bit, just use bt [esi],ecx
A small release, but nevertheless significant for 1k coders: Crinkler 2.2!
- Decompression code size for /TINYHEADER reduced by 6 bytes. Big thanks to TomCat for showing us the way to these optimizations! The one-byte LOOP optimization independently suggested by bitRAKE above is included in these 6 bytes.
- When a /TINYHEADER intro was recompressed, its static memory requirements would increase by 16MB every time. This bug is now fixed.
- Decompression code size for /TINYHEADER reduced by 6 bytes. Big thanks to TomCat for showing us the way to these optimizations! The one-byte LOOP optimization independently suggested by bitRAKE above is included in these 6 bytes.
- When a /TINYHEADER intro was recompressed, its static memory requirements would increase by 16MB every time. This bug is now fixed.
All Crinkler versions after 2.0a crash under wine :(
E.g. crash for 2.2:
Tried wine-4.0.1, wine-4.12.1
Crinkler 2.0a works fine with the same arguments and object files.
E.g. crash for 2.2:
Quote:
Unhandled exception: page fault on read access to 0x00b3959d in 64-bit code (0x0000000140024770).
002c:fixme:dbghelp:interpret_function_table_entry PUSH_MACHFRAME 6
002c:fixme:dbghelp:interpret_function_table_entry PUSH_MACHFRAME 6
Register dump:
rip:0000000140024770 rsp:000000000023d380 rbp:0000000000733b50 eflags:00010297 ( R- -- I S -A-P-C)
rax:0000000000b3959d rbx:0000000000005a4d rcx:0000000000733b50 rdx:0000000000077ff4
rsi:0000000000733b50 rdi:0000000000000000 r8:ffffffffff7b9e8b r9:0000000000000003 r10:0000000000405a4d
r11:0000000000000003 r12:00000000002f3428 r13:0000000000000000 r14:0000000000733b50 r15:0000000000000060
Stack dump:
0x000000000023d380: 0000000000005a4d 00000000007339e0
0x000000000023d390: 000000000023d300 0000000000000000
0x000000000023d3a0: 0000000000000000 00000000002f33b0
0x000000000023d3b0: 0000000000000000 0000000000000000
0x000000000023d3c0: 0000000000000000 0000000000000000
0x000000000023d3d0: 000000000023d429 000000014002437e
0x000000000023d3e0: 00000000002f3448 00000000002f3428
0x000000000023d3f0: 0000000000733b50 0000000000733bb0
0x000000000023d400: 0000000000000000 0000000000000000
0x000000000023d410: 0000000000000040 0000000140060fe7
0x000000000023d420: 0000000000000000 0000000000000000
0x000000000023d430: 0000000000000000 0000000000000000
Backtrace:
=>0 0x0000000140024770 in crinkler (+0x24770) (0x0000000000733b50)
1 0x000000014002437e in crinkler (+0x2437d) (0x000000000023d429)
2 0x00000001400226d6 in crinkler (+0x226d5) (0x000000000023d590)
3 0x00000001400063db in crinkler (+0x63da) (0x000000000023dba9)
4 0x000000014008d244 in crinkler (+0x8d243) (0x0000000140029691)
Tried wine-4.0.1, wine-4.12.1
Crinkler 2.0a works fine with the same arguments and object files.
Really should submit a bug report to Wine and not to Crinkler. :) They do fix their stuff.
I investigated the Wine crash.
To implement forwarded RVA imports by link-time forwarding, Crinkler needs to inspect the actual DLL inported, rather than just the LIB file. In versions prior to 2.1, we just loaded the DLL using LoadLibrary, but this does not work with the 64-bit Crinkler, since you can't load a 32-bit DLL into a 64-bit process. For this reason, we implemented our own DLL loading and parsing function in 2.1.
Apparently, the DLLs in Wine (at least kernel32.dll and user32.dll) have a different structure, which causes this function to crash.
This issue could possibly be fixed in Crinkler, but there is a workaround for running the current Crinkler on Wine:
Copy kernel32.dll and user32.dll from C:\Windows\SysWOW64 on a real Windows to the current directory (from which you run Crinkler) on your Wine machine. This will cause Crinkler to find those DLLs instead of the ones in the Wine installation. This is actually a good idea in any case, since this will prevent picking up bogus forwarded RVA imports from the Wine DLLs.
To implement forwarded RVA imports by link-time forwarding, Crinkler needs to inspect the actual DLL inported, rather than just the LIB file. In versions prior to 2.1, we just loaded the DLL using LoadLibrary, but this does not work with the 64-bit Crinkler, since you can't load a 32-bit DLL into a 64-bit process. For this reason, we implemented our own DLL loading and parsing function in 2.1.
Apparently, the DLLs in Wine (at least kernel32.dll and user32.dll) have a different structure, which causes this function to crash.
This issue could possibly be fixed in Crinkler, but there is a workaround for running the current Crinkler on Wine:
Copy kernel32.dll and user32.dll from C:\Windows\SysWOW64 on a real Windows to the current directory (from which you run Crinkler) on your Wine machine. This will cause Crinkler to find those DLLs instead of the ones in the Wine installation. This is actually a good idea in any case, since this will prevent picking up bogus forwarded RVA imports from the Wine DLLs.
By the way, a nice trick I found for debugging a crash in Wine:
Copy the .pdb file for the program you are trying to run next to the .exe file. This will cause the Wine crash dump to include symbol information.
Copy the .pdb file for the program you are trying to run next to the .exe file. This will cause the Wine crash dump to include symbol information.
Quote:
Any chance it will be open sourced? I would like to make a version for 64-bit executables: https://board.flatassembler.net/topic.php?t=21094
You may be interested in XLINK, the open source compressing linker I wrote for DOS COM files https://github.com/negge/xlink.
It uses the same PAQ1 compression algorithm as crinkler and I am working on adding support for linux binaries. I gave a talk about this project at linux.conf.au earlier this year https://www.youtube.com/watch?v=J5WX-wN_RKY.
Patches welcome!
!
Hello!
I would like to say that this is a very cool product! Thanks to those people who develop it!
But I have some problem, I wrote dll, pointed out /CRINKLER /ENTRY:DllMain msvcrt.lib
but when I call DllMain from any program, it does not find an entry point
help me please!
I would like to say that this is a very cool product! Thanks to those people who develop it!
But I have some problem, I wrote dll, pointed out /CRINKLER /ENTRY:DllMain msvcrt.lib
but when I call DllMain from any program, it does not find an entry point
help me please!
Interesting request, producing a DLL using Crinkler. I went on a little investigation to figure out if this was (reasonably easily) achievable. There are a bunch of issues:
- Crinkler produces an EXE file, rather than a DLL. The format is almost the same - there's just a flag that indicates that the file is a DLL. Adding an option to Crinkler to set that would be very easy. It can also be easily patched in manually, by changing 00 to 20 at offset 1B in the output file. So far so good.
- To produce an export table containing the exported functions, use the /EXPORT option. This process is different from how exports are usually specified (since the intended use case is somewhat different). Every exported function needs its own /EXPORT option on the Crinkler commandline.
- The DllMain function is not published though an export, but rather through the entry point. Specifying this function for /ENTRY is indeed the correct way. It is called automatically when the DLL is loaded (by LoadLibrary or via import) and can't be called explicitly.
- EXE files produced by Crinkler use a fixed base address of 0x400000 (the usual base address for EXE files) and do not include relocation information. Thus, to be loadable into the same address space as the host program, that program must be placed somewhere else. This can be achieved via the /BASE option to the Visual Studio linker. To further shrink the amount of address space taken up by the Crinkler-produced DLL, specify a small value to /HASHSIZE.
- The Crinkler entry point performs decompression and function import and then calls the user-specified entry point. This code depends on certain registers having specific values at entry (specifically, EBX points to the PEB), which might not hold for DLL entry point invocation. Also, this code does not save the registers required by the calling convention, which could mess up the calling function.
- The export table produced by Crinkler is part of the compressed data, so it is not available before the decompression code has run. I don't think anything actually tries to look at the export table before calling the DLL entry point (which will do the decompression), but you never know...
Anyway, I tried to see if I could produce a DLL with Crinkler this way and load it. It got as far as actually loading the DLL into memory and calling the entry point (decompression code), but somewhere between here and being fully decompressed, something goes wrong, and the initilization fails. I have not been able to locate it further.
- Crinkler produces an EXE file, rather than a DLL. The format is almost the same - there's just a flag that indicates that the file is a DLL. Adding an option to Crinkler to set that would be very easy. It can also be easily patched in manually, by changing 00 to 20 at offset 1B in the output file. So far so good.
- To produce an export table containing the exported functions, use the /EXPORT option. This process is different from how exports are usually specified (since the intended use case is somewhat different). Every exported function needs its own /EXPORT option on the Crinkler commandline.
- The DllMain function is not published though an export, but rather through the entry point. Specifying this function for /ENTRY is indeed the correct way. It is called automatically when the DLL is loaded (by LoadLibrary or via import) and can't be called explicitly.
- EXE files produced by Crinkler use a fixed base address of 0x400000 (the usual base address for EXE files) and do not include relocation information. Thus, to be loadable into the same address space as the host program, that program must be placed somewhere else. This can be achieved via the /BASE option to the Visual Studio linker. To further shrink the amount of address space taken up by the Crinkler-produced DLL, specify a small value to /HASHSIZE.
- The Crinkler entry point performs decompression and function import and then calls the user-specified entry point. This code depends on certain registers having specific values at entry (specifically, EBX points to the PEB), which might not hold for DLL entry point invocation. Also, this code does not save the registers required by the calling convention, which could mess up the calling function.
- The export table produced by Crinkler is part of the compressed data, so it is not available before the decompression code has run. I don't think anything actually tries to look at the export table before calling the DLL entry point (which will do the decompression), but you never know...
Anyway, I tried to see if I could produce a DLL with Crinkler this way and load it. It got as far as actually loading the DLL into memory and calling the entry point (decompression code), but somewhere between here and being fully decompressed, something goes wrong, and the initilization fails. I have not been able to locate it further.
Hello!
Crinkler fails to link lib files produced by rustc. Any idea why this could be?
Thanks!
Crinkler fails to link lib files produced by rustc. Any idea why this could be?
Thanks!
Not knowing what you meant by "fails to", I tried it out to see what the issue was. With no_std it seems to work fine. I have:
which I compile with:
and link with (omitting LIBPATH):
The resulting out.exe is 484 bytes and prints correctly.
What is the problem you are encountering?
Code:
#![no_std]
#![no_main]
use core::panic::PanicInfo;
#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
loop {}
}
#[no_mangle]
pub extern "C" fn main() {
unsafe {
printf("Hello World!\n\0".as_ptr());
}
}
extern "cdecl" {
pub fn printf(text: *const u8);
}
which I compile with:
Code:
rustc no_std_hello.rs --emit obj --target i686-pc-windows-msvc -C panic=abortand link with (omitting LIBPATH):
Code:
crinkler no_std_hello.o /ENTRY:main kernel32.lib user32.lib msvcrt_old.libThe resulting out.exe is 484 bytes and prints correctly.
What is the problem you are encountering?
Ah, now I noticed that you were asking about lib files from rustc.
Indeed those are rejected by Crinkler with the the message
This is definitely worth looking into.
Anyway, I learned a lot from my detour with the no_std example above. Thanks for bringing it up.
I wonder when we will see the first 4k intro coded in Rust? ;)
Indeed those are rejected by Crinkler with the the message
Code:
error LNK: Unsupported file typeThis is definitely worth looking into.
Anyway, I learned a lot from my detour with the no_std example above. Thanks for bringing it up.
I wonder when we will see the first 4k intro coded in Rust? ;)
impressive
.
Quote:
I wonder when we will see the first 4k intro coded in Rust? ;)
I recently released my first 4K intro which was fully written in Rust and compressed with Crinkler. To my knowledge this is the first 4K intro in Rust. The biggest difficulty in using crinkler with Rust was figuring out how to get the rust compiler to output the right kind of obj files
The intro took 1st place in the new school intro competition https://www.pouet.net/prod.php?which=85924
Happy 15th anniversary to all Crinkler users!
We have two big pieces of news for you today.
First, as hinted a week ago, Crinkler is now open source! After lots of preparations, today we are relasing the Crinkler source code under the very permissive Zlib license. Find it on GitHub!
Second, we have a new version ready - Crinkler 2.3! The main improvements are:
- The size of /TINYHEADER reduced by another 3 bytes.
- Faster hash size optimization, especially with many cores.
- Error messages about symbols that cannot be found now include the name of the function containing the reference to that symbol.
- When using functions from msvcrt.dll, it is no longer necessary to supply an old msvcrt.lib. Crinkler now automatically links to msvcrt.dll, similarly to how the Microsoft linker automatically links to its corresponding runtime.
- For CONSOLE applications, when not specifying a custom entry point, Crinkler will insert a small mainCRTStartup which handles commandline arguments, similarly to what dnm221 requested a while back.
- The automatic msvcrt.dll linking and entry point can be disabled by using the /NODEFAULTLIB option.
- Added support for legacy format lib files with only the first linker member. Files like this are emitted from rustc, for example. This fixes the problem reported by p2mate above.
- When an imported DLL contains no export table (which is the case for some Wine DLLs), Crinkler will no longer crash. Instead, it will report an error, instructing the user to supply genuine Windows DLLs when crinkling on Wine.
We have two big pieces of news for you today.
First, as hinted a week ago, Crinkler is now open source! After lots of preparations, today we are relasing the Crinkler source code under the very permissive Zlib license. Find it on GitHub!
Second, we have a new version ready - Crinkler 2.3! The main improvements are:
- The size of /TINYHEADER reduced by another 3 bytes.
- Faster hash size optimization, especially with many cores.
- Error messages about symbols that cannot be found now include the name of the function containing the reference to that symbol.
- When using functions from msvcrt.dll, it is no longer necessary to supply an old msvcrt.lib. Crinkler now automatically links to msvcrt.dll, similarly to how the Microsoft linker automatically links to its corresponding runtime.
- For CONSOLE applications, when not specifying a custom entry point, Crinkler will insert a small mainCRTStartup which handles commandline arguments, similarly to what dnm221 requested a while back.
- The automatic msvcrt.dll linking and entry point can be disabled by using the /NODEFAULTLIB option.
- Added support for legacy format lib files with only the first linker member. Files like this are emitted from rustc, for example. This fixes the problem reported by p2mate above.
- When an imported DLL contains no export table (which is the case for some Wine DLLs), Crinkler will no longer crash. Instead, it will report an error, instructing the user to supply genuine Windows DLLs when crinkling on Wine.
Obligatory thumb! Now, question: if a named .code section ends with JMP to the beginning of another section, can Crinkler eliminate that jump automagically?
This is certainly a feature we could add.
We already have a similar feature specifically for the end of the import section, where it will insert a JMP if the entry point is not adjacent. We could extend this mechanism to the other sections and then detect and chop off trailing JMP instructions (when the option is enabled). That should do it.
It should interact well with the section reordering, since the linking step for each potential ordering will be able to take these JMP instructions into account and thus will likely prefer placing sections such that the JMP instructions are eliminated.
We already have a similar feature specifically for the end of the import section, where it will insert a JMP if the entry point is not adjacent. We could extend this mechanism to the other sections and then detect and chop off trailing JMP instructions (when the option is enabled). That should do it.
It should interact well with the section reordering, since the linking step for each potential ordering will be able to take these JMP instructions into account and thus will likely prefer placing sections such that the JMP instructions are eliminated.
This sounds great! I have some helper functions and other functions tail call them using jmps... All functions are in their own sections. I can of course try to eliminate the JMP myself by placing the helper function after one of the calling functions, but the trouble is; depending on %defines, some of the calling functions might or might not be included. So, it's a bit tricky to figure out where to place the helper function to eliminate that JMP. Having crinkler do it for me would be fantastic! Also, this would give Crinkler the freedom to choose the best place to eliminate that jump, in terms of compression ratio.
Has caused so much carnage; the only other demotool that can claim a similar influence is probably FT2.
Windows Defender makes different with and without /TINYHEADER. Its nice thought too see it strip of 59 bytes or something. Altho, it would be nice to have a turnaround such that lazy ppl get away with adding an exception for virus detection.
Crinkler just rulez so hard!
submit changes
if this prod is a fake, some info is false or the download link is broken,
do not post about it in the comments, it will get lost.
instead, click here !