Help with 4k
category: offtopic [glöplog]
Code:
static const char *sync_track_name(const char *base, const char *name)
{
static char temp[FILENAME_MAX];
strncpy(temp, base, sizeof(temp) - 1);
temp[sizeof(temp) - 1] = '\0';
strncat(temp, "_", sizeof(temp) - 1);
strncat(temp, name, sizeof(temp) - 1);
strncat(temp, "_", sizeof(temp) - 1);
strncat(temp, "data", sizeof(temp) - 1);
return temp;
}
void sync_save_tracks(const struct sync_device *d)
{
FILE *file = fopen("sync.h","wt");
int len = 0;
FILE *fileread = NULL;
int count;
int i;
for (i = 0; i < (int)d->data.num_tracks; ++i) {
const struct sync_track *t = d->data.tracks[i];
save_track(t, sync_track_path(d->base, t->name));
//read trackfile
fprintf(file,"//data from rocket track: %s\n",sync_track_path(d->base, t->name));
fileread = fopen(sync_track_path(d->base, t->name),"rb");
fseek( fileread, 0, SEEK_END);
len = ftell(fileread);
fseek(fileread, 0, SEEK_SET);
count = 0;
//write data
fprintf(file,"unsigned char %s[%d] = {\n\t",sync_track_name(d->base, t->name),len);
while (!feof(fileread))
{
unsigned char ch = fgetc(fileread);
fprintf(file,"0x%02x,",ch);
count++;
if ((count & 15) == 0)
fprintf(file,"\n\t");
}
fprintf(file,"\n};\n// end of %s\n",sync_track_path(d->base, t->name));
fprintf(file,"\n");
fclose(fileread);
}
fclose(file);
}
My current code. Dumps GNU Rocket track files to memory arrays, which I then load using a very thin memio wrapper based on the FILE* based stdio functions. Works brilliantly so far.
mudlord:
I noticed that you changed the fix point values storage to floats.
I actually have originally implemented the tiny player to have different data array for each field of the event (time, position, type) - the idea is to have a better data compression rates.
Having the data in a arrays, I tried several things like storing deltas, fix-point, even tweaking values to have more 0 bits (ie 1024 instead of 1023). All those help but the major compression advantage came from using fix point for value storage, those get compressed much better compare to float arrays.
I noticed that you changed the fix point values storage to floats.
I actually have originally implemented the tiny player to have different data array for each field of the event (time, position, type) - the idea is to have a better data compression rates.
Having the data in a arrays, I tried several things like storing deltas, fix-point, even tweaking values to have more 0 bits (ie 1024 instead of 1023). All those help but the major compression advantage came from using fix point for value storage, those get compressed much better compare to float arrays.
My code is intended for raw unmodified GNU Rocket, I didn't do any mods to use fixedpoint, etc.
IIRC, spookysys just added some editor-support to truncate mantissa bits in the editor to save space in Texas. I'm sure something like that could be done upstream as well.
I did try to crack out some code. It's not super-useful in the current state, but it some kind of sketch at least.
While post-release squeezing Dodecaplicit below 10k (just for future stuff - cut ~1kb from the rocket data alone compared to the already binary representation used back then at release), I took another approach than TLM, which is more flexible and convenient (essentially the same API as the standard one), maybe compressing a bit better than what i can see in the code snippet, but the flexibility is obviously also costing some bytes. And it's not so much a code generator as TLMs.
So my version is meant for 8-64k, while TLMs is the 4k suited version (I would probably still go completely custom for 4k, but that's another story)
I'm storing per element type (keys, types, values), delta encoding keys (actually types are packed into the 16bit keys after that), while values are quantized floats. Quantization is something you specify per variable at creation time, and not in neither editor nor player code (basicly just the import from the editor).
Variable names (and quantization) are #defined away in player version, and instead only the order of initialization is used.
So I got a compact format that compresses well, smaller player code and except the optional quantization bits and the data pointer the same user code as the "demo" version.
Not too happy about the current state of the source though - it has become a bit too much #define hell, should probably move the player to it's own file..
So my version is meant for 8-64k, while TLMs is the 4k suited version (I would probably still go completely custom for 4k, but that's another story)
I'm storing per element type (keys, types, values), delta encoding keys (actually types are packed into the 16bit keys after that), while values are quantized floats. Quantization is something you specify per variable at creation time, and not in neither editor nor player code (basicly just the import from the editor).
Variable names (and quantization) are #defined away in player version, and instead only the order of initialization is used.
So I got a compact format that compresses well, smaller player code and except the optional quantization bits and the data pointer the same user code as the "demo" version.
Not too happy about the current state of the source though - it has become a bit too much #define hell, should probably move the player to it's own file..
Psycho: Cool, I'd love to see the code.
Another reasonably small approach is how spooky did the replayer in Texas: at init-time, expand all sync-data to piecewise linear segments sampled at a fixed frequency. He used 50hz, so that's some pretty huge buffers, but I think by doing piece-wise quadrics instead, it only really needs one sample pr row. That gives you a simple lookup for a simple evaluate function, give random row-access without worries. But I do suspect that it costs a few bytes more than interpolating once per frame.
And besides, I must admit: even when size isn't an issue, I very rarely do any thing else than reading at the current row.
Another reasonably small approach is how spooky did the replayer in Texas: at init-time, expand all sync-data to piecewise linear segments sampled at a fixed frequency. He used 50hz, so that's some pretty huge buffers, but I think by doing piece-wise quadrics instead, it only really needs one sample pr row. That gives you a simple lookup for a simple evaluate function, give random row-access without worries. But I do suspect that it costs a few bytes more than interpolating once per frame.
And besides, I must admit: even when size isn't an issue, I very rarely do any thing else than reading at the current row.
I've found some bugs in kusma's usync implementation, here's a version that exports correct values (floats instead of decimals) and finds the right row during playback:
usync.c:
(I haven't used github before, so I don't really know how to contribute/patch. Please forgive this)
usync.c:
Code:
#include "usync.h"
#include <math.h>
#ifdef SYNC_PLAYER
static int usync_rows[SYNC_TRACK_COUNT];
float usync_values[SYNC_TRACK_COUNT];
void usync_update(float t)
{
int i, row = (int)floor(t);
for (i = 0; i < SYNC_TRACK_COUNT; ++i) {
int pos;
float mag, x, a, b, c, d;
/* empty tracks should not be neccesary! */
if (!sync_data_count[i]) {
usync_values[i] = 0.0f;
continue;
}
/* step forward until we're at the right key-frame */
while (usync_rows[i] < (sync_data_count[i] - 1) &&
row >= sync_data_rows[sync_data_offset[i] + usync_rows[i] + 1]) {
usync_rows[i]++;
}
pos = usync_rows[i] + sync_data_offset[i];
/* we need a segment to interpolate over */
if (usync_rows[i] == sync_data_count[i] - 1) {
usync_values[i] = sync_data_values[pos];
continue;
}
/* prepare coefficients for interpolation */
a = sync_data_values[pos];
mag = sync_data_values[pos + 1] - sync_data_values[pos];
switch (sync_data_type[pos]) {
case 0:
b = c = d = 0.0f;
break;
case 1:
b = mag;
c = d = 0.0f;
break;
case 2:
b = 0.0f;
c = 3 * mag;
d = -2 * mag;
break;
case 3:
b = d = 0.0f;
c = mag;
break;
}
/* evaluate function */
x = (t - sync_data_rows[pos]) / (sync_data_rows[pos + 1] - sync_data_rows[pos]);
usync_values[i] = a + (b + (c + d * x) * x) * x;
}
}
#else /* !defined(SYNC_PLAYER) */
#include <stdio.h>
#include "modified-rocket/sync.h"
#include "modified-rocket/device.h"
struct sync_device *usync_dev;
float usync_time = 0;
void usync_update(float t)
{
usync_time = t;
sync_update(usync_dev, (int)floor(t), &usync_cb, usync_data);
}
int usync_init(void)
{
usync_dev = sync_create_device("sync");
return sync_connect(usync_dev, "localhost", SYNC_DEFAULT_PORT);
}
void usync_export(void)
{
int i, j;
int offset = 0;
FILE *fp = fopen("E:\\blu-flame.org\\nordlicht2014-intro\\sync-data.h", "w");
if (!fp)
return;
/* header-guard */
fputs("#ifndef SYNC_DATA_H\n#define SYNC_DATA_H\n\n", fp);
fputs("enum sync_tracks {\n", fp);
for (i = 0; i < usync_dev->data.num_tracks; ++i) {
struct sync_track *t = usync_dev->data.tracks[i];
fprintf(fp, "\tSYNC_TRACK_%s = %d,\n", t->name, i);
}
fprintf(fp, "\tSYNC_TRACK_COUNT = %d\n", usync_dev->data.num_tracks);
fputs("};\n\n", fp);
fputs("static const int sync_data_offset[SYNC_TRACK_COUNT] = {\n", fp);
for (i = 0; i < usync_dev->data.num_tracks; ++i) {
struct sync_track *t = usync_dev->data.tracks[i];
fprintf(fp, "\t%d, /* track: %s */\n", offset, t->name);
offset += t->num_keys;
}
fputs("};\n\n", fp);
fputs("static const int sync_data_count[SYNC_TRACK_COUNT] = {\n", fp);
for (i = 0; i < usync_dev->data.num_tracks; ++i) {
struct sync_track *t = usync_dev->data.tracks[i];
fprintf(fp, "\t%d, /* track: %s */\n", t->num_keys, t->name);
}
fputs("};\n\n", fp);
fputs("static const int sync_data_rows[] = {\n", fp);
for (i = 0; i < usync_dev->data.num_tracks; ++i) {
struct sync_track *t = usync_dev->data.tracks[i];
fprintf(fp, "\t/* track: %s */\n", t->name);
for (j = 0; j < t->num_keys; ++j)
fprintf(fp, "\t%d,\n", t->keys[j].row);
}
fputs("};\n\n", fp);
fputs("static const float sync_data_values[] = {\n", fp);
for (i = 0; i < usync_dev->data.num_tracks; ++i) {
struct sync_track *t = usync_dev->data.tracks[i];
fprintf(fp, "\t/* track: %s */\n", t->name);
for (j = 0; j < t->num_keys; ++j)
fprintf(fp, "\t%.6f,\n", t->keys[j].value);
}
fputs("};\n\n", fp);
fputs("static const unsigned char sync_data_type[] = {\n", fp);
for (i = 0; i < usync_dev->data.num_tracks; ++i) {
struct sync_track *t = usync_dev->data.tracks[i];
fprintf(fp, "\t/* track: %s */\n", t->name);
for (j = 0; j < t->num_keys; ++j)
fprintf(fp, "\t%d,\n", t->keys[j].type);
}
fputs("};\n\n", fp);
fputs("#endif /* !defined(SYNC_DATA_H) */\n", fp);
fclose(fp);
}
#endif
(I haven't used github before, so I don't really know how to contribute/patch. Please forgive this)
Of course, the hardcoded paths should be replaced accordingly... ;)
Thanks! I wasn't aware anyone tried to use this version yet, so it's cool to see patches :)