Differences between writeStringToMemory and stringToUTF8

[kalwalt]

I have a project which was using writeStringToMemory with an old obsolete Emscripten version, as i upgraded to stringToUTF8 the code seems to act in a different way, have you had any issue with that? If you need i can post my code and explain the issue.

[sbc100]

It looks like writeStringToMemory is almost the same as stringToUTF8 except for two things:

1. writeStringToMemory as a third argument called dontAddNull which means that string won't be null terminated.
2. stringToUTF8 has a third argument maxBytesToWrite which allows you the limit the number of bytes written to memory.

Otherwise they looks the same and writeStringToMemory is implemented in terms of stringToUTF8:

emscripten/src/library_legacy.js

Lines 48 to 61 in bc2123f

	$writeStringToMemory: (string, buffer, dontAddNull) => {
	warnOnce('writeStringToMemory is deprecated and should not be called! Use stringToUTF8() instead!');
	var /** @type {number} */ lastChar, /** @type {number} */ end;
	if (dontAddNull) {
	// stringToUTF8 always appends null. If we don't want to do that, remember the
	// character that existed at the location where the null will be placed, and restore
	// that after the write (below).
	end = buffer + lengthBytesUTF8(string);
	lastChar = HEAP8[end];
	}
	stringToUTF8(string, buffer, Infinity);
	if (dontAddNull) HEAP8[end] = lastChar; // Restore the value under the null character.
	},

[kalwalt]

Thank you @sbc100 for the answer, i will post my code when i will be on my PC. I don't see any difference between the two, if you don't use the third argument.

[kalwalt]

Sorry for the delay, I hadn't time to replay; I will try to summarize the problem and how i solved:

I developed a simple app Nft-Marker-Creator-App to create NFT (Natural Feature Tracking) markers to be used in projects like AR.js, jsartoolkit5, jsartoolkitNFT and ARnft to cite few of them. All about WebAR. The original code was by @Carnaux he did a great job, in the past he added the feature to compress the three single files(.fset, .iset and .fset3) into one only file called .zft into one. It works fine but when i upgraded to a more recent emsdk version, the compressed file was malformed ( itcan not be read by the WebAR application in the browser).
The C compressZip function does his job:

#include <emscripten.h>

#ifdef _WIN32
#include <Windows.h>
#else
#include <sys/stat.h>
#endif

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <AR/ar.h>
#include <zlib/zlib.h>

static const char *zipname = "/tempBinFile.bin";
extern "C"
{


    uint compressZip(char *src, int srclen)
    {
        FILE *fp;


        char *b = new char[srclen];


        printf("Uncompressed size is: %lu", strlen(src));
        printf("\n----------\n");


        z_stream defstream;
        defstream.zalloc = Z_NULL;
        defstream.zfree = Z_NULL;
        defstream.opaque = Z_NULL;
        defstream.avail_in = (uInt)srclen;
        defstream.next_in = (Bytef *)src;
        defstream.avail_out = (uInt)srclen;
        defstream.next_out = (Bytef *)b;


        deflateInit(&defstream, Z_BEST_COMPRESSION);
        deflate(&defstream, Z_FINISH);
        deflateEnd(&defstream);


        printf("Compressed size is: %lu\n", strlen(b));


        fp = fopen(zipname, "wb");
        fwrite(b, defstream.total_out, 1, fp);
        fclose(fp);


        delete[] b;


        return 0;
    }

and used this flag -s EXPORTED_FUNCTIONS=["_compressZip,_malloc,_free"] -s EXPORTED_RUNTIME_METHODS=["FS,stringToUTF8"] for emscripten
this was the old javascript code to compress the generated files into one, It use writeStringToMemory because an older emscripten version:

    let obj = {
      iset: iset.toString("hex"),
      fset: fset.toString("hex"),
      fset3: fset3.toString("hex"),
    };


    let strObj = JSON.stringify(obj);


    let StrBufferZip = Module._malloc(strObj.length + 1);
    Module.stringToUTF8(strObj, StrBufferZip);


    Module._compressZip(StrBufferZip, strObj.length);


    let contentBin = Module.FS.readFile("tempBinFile.bin");


    // fs.writeFileSync(path.join(__dirname, '/output/') + fileName + ".zft", contentBin);
    fs.writeFileSync(outputPath + fileName + ".zft", contentBin);

original code on github:
https://github.com/webarkit/Nft-Marker-Creator-App/blob/59fc53a9b718b280c82abe6024d43db1922afa3a/src/NFTMarkerCreator.js#L207-L223

This after upgrading to a newer emscripten version:
```
let obj = {
iset: iset.toString("hex"),
fset: fset.toString("hex"),
fset3: fset3.toString("hex"),
};

let strObj = JSON.stringify(obj);


let lengthStrObj = Module.lengthBytesUTF8(strObj);


let StrBufferZip = Module._malloc(lengthStrObj + 1);


Module.stringToUTF8(strObj, StrBufferZip);


//Module._compressZip(StrBufferZip, strObj.length);
Module.compressZip(strObj, strObj.length);


let contentBin = Module.FS.readFile("tempBinFile.bin");


fs.writeFileSync(path.join(__dirname, "/output/") + fileName + ".zft", contentBin);

https://github.com/webarkit/Nft-Marker-Creator-App/blob/92e80984a443156b01a2218e6ce3b084594a0d72/src/NFTMarkerCreator.js#L207-L226

I solved changing the compressZip function:

```c++
#include "zlib/zlib.h"
#include <AR/ar.h>

static const char *zipname = "/tempBinFile.bin";

int compressZip(char *src, int srclen) {
  FILE *fp;

  char *b = new char[srclen];

  printf("Uncompressed size is: %lu", strlen(src));
  printf("\n----------\n");

  z_stream defstream;
  defstream.zalloc = Z_NULL;
  defstream.zfree = Z_NULL;
  defstream.opaque = Z_NULL;
  defstream.avail_in = (uInt)srclen;
  defstream.next_in = (Bytef *)src;
  defstream.avail_out = (uInt)srclen;
  defstream.next_out = (Bytef *)b;

  deflateInit(&defstream, Z_BEST_COMPRESSION);
  deflate(&defstream, Z_FINISH);
  deflateEnd(&defstream);

  printf("Compressed size is: %lu\n", strlen(b));

  fp = fopen(zipname, "wb");
  fwrite(b, defstream.total_out, 1, fp);
  fclose(fp);

  delete[] b;

  return 0;
}

then with EMSCRIPTEN_BINDINGS:

#include <emscripten/bind.h>
#include <emscripten/val.h>
#include <emscripten.h>

using namespace emscripten;

int createNftDataSet_em(emscripten::val imgData, float dpiIn, int xsizeIn,
                        int ysizeIn, int ncIn, std::string cmdStr) {
  std::vector<uint8_t> idata =
      emscripten::convertJSArrayToNumberVector<uint8_t>(imgData);
  return createNftDataSet(idata.data(), dpiIn, xsizeIn, ysizeIn, ncIn,
                          &cmdStr[0]);
}

int compressZip_em(std::string srcStr, int srclen) {
  return compressZip(&srcStr[0], srclen);
}

EMSCRIPTEN_BINDINGS(markerCreator_bindings) {
  function("createNftDataSet", &createNftDataSet_em);
  function("compressZip", &compressZip_em);
};

of course i added a --bind flags to the emcc commands and after that the issue was solved.