diff --git a/README.md b/README.md index 4ece7ec..6fa5add 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,8 @@ The following QuickBMS commands are currently supported by the langauge server. * [x] `[Else]` * [x] `EndIf` * [ ] `GetCT VAR TYPE CHAR [FILENUM]` -* [ ] `ComType ALGO [DICT] [DICT_SIZE]` +* [x] `ComType ALGO [DICT] [DICT_SIZE]` + * [ ] Support optional arguments * [x] `ReverseShort VAR [ENDIAN]` * [x] `ReverseLong VAR [ENDIAN]` * [x] `ReverseLongLong VAR [ENDIAN]` diff --git a/src/server/keyword_docs/comtype.txt b/src/server/keyword_docs/comtype.txt new file mode 100644 index 0000000..b619279 --- /dev/null +++ b/src/server/keyword_docs/comtype.txt @@ -0,0 +1,1081 @@ +ComType ALGO [DICT] [DICT_SIZE] + + It selects a specific compression algorithm to use with the + Clog command. + It's also possible to choose a number as ALGO, this is a + feature coming from my project for a compression scanner and + brute-forcer able to guess the possible algorithm of an + unknown raw compressed data block by simply trying every + decompressor on the input (do NOT use it if you don't know what + you are doing! this is NOT offzip!): + http://aluigi.org/bms/comtype_scan2.bat + http://aluigi.org/bms/comtype_scan2.bms + comtype_scan2.bat comtype_scan2.bms input_file output_folder + comtype_scan2.bat comtype_scan2.bms input_file output_folder uncompressed_size + Note that some algorithms may work on Windows only. + if ALGO is "?", the user is prompted to type the desired + algorithm name. + + Arguments: + ALGO copy, simple copy that useful in some rare cases + with data encrypted with block ciphers like AES + and blowfish so use comtype copy and encryption + zlib, RFC1950 (windowbit 15, data starts with 'x') + DICT supported + if DICT starts with "Z_FULL_FLUSH" + inflate/deflate will use Z_FULL_FLUSH + deflate, RFC1951 (windowbit -15) used in ZIP files + DICT supported + if DICT starts with "Z_FULL_FLUSH" + inflate/deflate will use Z_FULL_FLUSH + lzo1x + DICT supported + lzo1a till lzo2a, LZO (the most used is lzo1x!) + DICT supported + lzss, with default configuration (window of 4096) + this particular algorithm can be fully configured + setting the EI, EJ and P fields plus another + number rarely used: + EI, EJ, P, rless, init_chr + for setting them it's enough to use a DICT equal + to something like "12 4 2" which means + EI:12 (N:4096), EJ:4 (F:18), P:2 + use a negative init_chr for different window + slide customization like -1 for Tales of Vesperia + the default character of lzss is a space (0x20), + but you can use the ALGO lzss0 or the dictionary + "12 4 2 2 0" to use 0x00 + lzx, used by the old unlzx tool and on Amiga + gzip, automatic handling of gzip data, in this case + the uncompressed size is ignored and calculated + automatically so in CLog use ZSIZE ZSIZE + pkware, algorithm also known as blast, explode, + implode or DCL + lzma, 5 bytes + lzma (in some cases you may need to + use ZSIZE + 5) + lzma86head, 5 bytes + 8 bytes (size) + lzma + lzma86dec, 1 byte + 5 bytes + lzma (in some cases + you may need to use ZSIZE + 5) + lzma86dechead, 1 + 5 + 8 bytes (size) + lzma + lzmaefs, the format implemented in ZIP + bzip2 + XMemDecompress, (xmemlzx) Xbox 360 LZX algorithm of + xcompress.lib use DICT to specify a custom + WindowSize and CompressionPartitionSize like + "131072 524288", the algorithm automatically + idenfities and extracts the LZXTDECODE and + LZXNATIVE files generated by xbcompress.exe + (0x0FF512ED / 0x0FF512EE) + hex, from "01234567" to bytes: 0x01 0x23 0x45 0x67 + base64, from "aGVsbG8=" to "hello", also supports + the Gamespy and URL chars + uudecode + ascii85 + yenc + COM_LZW_Decompress, used in Vietcong + milestone_lzw, lzw used in the Milestone games + lzxcab, lzx used in cab files (libmspack 21 0) + lzxchm, lzx used in chm files (libmspack 16 2) + rlew, 16 bit RLE algorithm used in AIM Racing + lzjb, a compression used in a file system for *nix + sfl_block, expand_block from iMatix SFL + sfl_rle, expand_rle from iMatix SFL + sfl_nulls, expand_nulls from iMatix SFL + sfl_bits, expand_bits from iMatix SFL + lzma2, 1 bytes + lzma2 + lzma2_86head, 1 bytes + 8 bytes (size) + lzma2 + lzma2_86dec, 1 byte + 1 bytes + lzma2 + lzma2_86dechead, 1 + 1 + 8 bytes (size) + lzma2 + nrv2b, UCL + nrv2d, UCL + nrv2e, UCL + huffboh, used in the Asura engine + uncompress, lzw used in the compress utility, use + dict for bits (the lzw data starts from offset 3 + of the .Z files) + dmc, Dynamic Markov Compression (DMC) + lzhuf, aka LZH/LHA + lzari + rle7 + rle0 + rle + rlea, another generic rle decompressor + use DICT to choose the escape char + bpe, byte pair encoding + quicklz + q3huff, Adaptive Huffman used in the Quake 3 engine + unmeng, algorithm used in DreamKiller + lz2k, used in various games of Traveller's Tales + darksector, lzfx used in the game Dark Sector + mszh, used in the LossLess Codec Library + un49g, used in the games of 49Games + unthandor, used in the old game Thandor + doomhuff, huffman used in doom, hexen, skulltag and + other doom ports, the DICT field can be used to + specify a custom HuffFreq table (256 floats) + aplib + tzar_lzss, used in Tzar of HaemimontGames + DICT must contain the name of the variable with + the algorithm number to use, example: + ComType tzar_lzss MYVAR + lzf, aka fastlz + clz77, http://compressions.sf.net/about.html + lzrw1 + dhuff, Huffman Decompression in LDS ("lossless + datacompression sources" kit 1.1) + fin, from LDS + lzah (not tested) + lzh12, aka -lh4- + lzh13, aka -lh5- + grzip, aka GRZipII + ckrle, Chilkat RLE + quad, raw data without the first 32bit field + balz, raw data without the first 9 bytes from files + deflate64 + shrink (not tested) + z-base-32 + base32hex + base32crockford + base32nintendo + base???, if ALGO starts with "base" then will be + taken its subsequent number (for example 32 if it + is "base32") and used for the conversion. the + function supports ANY base from 2 to 256. + for bases larger than 64 will be used a table + starting from byte 0x00 so base128 will have a + charset from 0 to 0x7f + brieflz + paq6, raw data block + shcodec + hstest_hs_unpack (never tested, may be removed) + hstest_unpackc (never tested, may be removed) + sixpack (never tested) + ashford (never tested, may be removed) + jcalg + jam + lzhlib + srank + zzip + scpack + DICT supported (for the SCPACK_TABLE field) + rle3 + bpe2 + bcl_huf, Basic Compression Library + bcl_lz, Basic Compression Library + bcl_rice, Basic Compression Library + you must use DICT to specify the format (1 to 8) + bcl_rle, Basic Compression Library + bcl_sf, Basic Compression Library + scz + szip + ppmd, ppmd var.i rev.1 with ZIP specifics so 2 + bytes of info followed by the compressed data + ppmdi_raw + ppmdg + ppmdg_raw, requires DICT "SASize MaxOrder" + ppmdh + ppmdh_raw, requires DICT "SASize MaxOrder" + ppmdj + ppmdj_raw, requires DICT "SASize MaxOrder CutOff" + sr3c + huffmanlib + sfastpacker, smart+simple mode + sfastpacker2, smart-mode only + dk2, RefPack used in Dungeon Keeper 2 and other + Bullfrog/EA games + lz77wii, (use input size as output size in clog) + lz77wii_raw10, tag 0x10 lz77 + darkstone, lz77 used in the game DarkStone + sfl_block_chunked, as sfl_block with automatic + handling of the chunks if used + yuke_bpe, used in the PS2 games developed by Yuke + stalker_lza, used in STALKER, use the output size + equal to the compressed one (handled by quickbms) + prs_8ing + puyo_cnx, raw compressed data from offset 0x10 + puyo_cxlz, raw compressed data from offset 0x8 + puyo_lz00, raw compressed data from offset 0x32 + puyo_lz01, raw compressed data from offset 0x10 + puyo_lzss, raw compressed data from offset 0x4 + puyo_onz, raw compressed data from offset 0x4 + puyo_prs + falcom + cpk, used by the CRI developers (LAYLA) + bzip2_file, exactly like bzip2 but it calculates + the output size, use ZSIZE ZSIZE in clog + lz77wii_raw11, tag 0x11 lzss + lz77wii_raw20, tag 0x20 huffman + lz77wii_raw30, tag 0x30 rle + lz77wii_raw40 + pglz, postgresql compression (headerless) + UnPackSLZ + slz_01, used in tri-ace slz type 1 + slz_02, used in tri-ace slz type 2 + slz_03, used in tri-ace slz type 2 + lzhl + d3101 + squeeze + lzrw3 + tdcb_ahuff + tdcb_arith + tdcb_arith1 + tdcb_arith1e + tdcb_arithn + tdcb_compand + tdcb_huff + tdcb_lzss, dict for INDEX_BIT_COUNT, + LENGTH_BIT_COUNT, DUMMY9, END_OF_STREAM + tdcb_lzw12 + tdcb_lzw15v + tdcb_silence + rdc + ilzr + dmc2 + diffcomp + lzr + lzs (aka mppc but NOT exactly the same!) + lzs_big (aka mppc_big but NOT exactly the same!) + mohlzss + mohrle + yaz0 (aka szs) + byte2hex + un434a + xxdecode + pack, the one supported in gzip + unzip_dynamic, automatic zlib/deflate and output + size, please note that while zlib is almost error + free due to the checksum at its end, deflate + doesn't guarantee a valid output so, even if it + uncompresses the data, it may be invalid. use + zlib_noerror if you are 100% sure that the input + is zlib or plain, and deflate_noerror for deflate + zlib_noerror, as zlib but doesn't quit in case of + errors and automatically reserves the output size + deflate_noerror, as above but for deflate + ppmdh + ppmdh_raw + rnc + rnc_raw + pak_explode, alone in the dark + KENS_Nemesis + KENS_Kosinski + KENS_Kosinski_moduled + KENS_Enigma + KENS_Saxman + dragonballz (STPZ/0DCS/0LCS/STPK archives, Spyke?) + NitroSDK (nitroCompLib) + zdaemon, like doomhuff but different freq table + skulltag, like doomhuff but different freq table + msf, headerless lzma same as lzma_0 + stargunner + ntcompress + crle + ctw + DACT_DELTA + DACT_MZLIB2 + DACT_MZLIB + DACT_RLE + DACT_SNIBBLE + DACT_TEXT + DACT_TEXTRLE + EXECUTE: + use DICT to specify the command to execute using + #INPUT# instead of input filename and #OUTPUT# + for the output one and the various variables like + you do for the Print command, example: + comtype EXECUTE "ctw.exe d #INPUT# #OUTPUT#" + comtype EXECUTE "ctw.exe d #INPUT# %NAME%" + clog "output.dat" 0 ZSIZE ZSIZE # no need SIZE + CALLDLL: + as above but allows to specify a calldll command + executed on input: "#INPUT#", "#INPUT_SIZE#", + "#OUTPUT#", "#OUTPUT_SIZE#" and %VAR% + full support also for pointers using the '&' or + '*' prefix like &MEMORY_FILE, &VAR, &#INPUT#, + &INPUT_SIZE + lz77_0 + lzbss + bpaq0 + lzpx, lzpxj + mar_rle + gdcm_rle + lzmat + dict + rep + lzp (it's a preprocessor, not a real compression) + elias_delta + elias_gamma + elias_omega + packbits + darksector_nochunks, aka lzf or lzfx + enet + eduke32, lzwuncompress + xu4_rle + rvl, lemur int compression + lzfu, MS RTF + lzfu_raw + xu4_lzw, Ultima 4 + he3, without the HE3\x0d signature and output size + iris, Ultima Online algorithms + iris_huffman + iris_uo_huffman + ntfs + pdb + COMPRLIB_SPREAD + COMPRLIB_RLE1 + COMPRLIB_RLE2 + COMPRLIB_RLE3 + COMPRLIB_RLE4 + COMPRLIB_ARITH + COMPRLIB_SPLAY + cabextract, it may be the same lzx of mspack + mrci + hd2_01 + hd2_08 + hd2_01raw + rtl_lznt1 + rtl_xpress, looks not supported by XP/7 + rtl_xpress_huff, looks not supported by XP/7 + prs + sega_lz77 + saint_seya, used for GMI compression + ntcompress30 + ntcompress40 + yakuza, used by SEGA CS1 team + lz4 (the algorithm of lz4hc is the same) + snappy + lunar_lz1 to lunar_lz19 + lunar_rle1 to lunar_rle4 + goldensun + luminousarc + lzv1 + fastlzah, it should be identical to lzf and lzfx + zax + shrinker + mmini_huffman + mmini_lz1 + mmini + clzw + lzham, use the dictionary to specify the following + fields: m_dict_size_log2, m_table_update_rate, + m_decompress_flags, m_table_max_update_interval, + m_table_update_interval_slow_rate + it will try to brute force the first 3 fields + lpaq8 + sega_lzs2, automatic CM/lzs2 and decompressed size + wolf + coreonline + mszip, "CK" included (from libmspack) + qtm, (from libmspack) + mslzss, (from libmspack) + mslzss1, (from libmspack) + mslzss2, (from libmspack) + kwaj, mslzh (from libmspack) + lzlib (lzip) + dflt + lzma_dynamic, automatic output size and automatic + scanning of any supported flag, so it should + blindly work against any compressed lzma input. + if it's not compressed, the input will be copied + on the output. + while "lzma" returns an error if the input buffer + doesn't have other compressed bytes + (LZMA_STATUS_NEEDS_MORE_INPUT), lzma_dynamic + gives the ok, this is useful with some rare cases + like Far Cry 3 (fat2_fat3.bms) + lzma2_dynamic, automatic output size + lzma2_efs + lzxcab_delta + lzxchm_delta + ffce + SCUMMVM1 -> SCUMMVM53, many algos used in Scummvm + lzs_unzip, PSP_Nanoha + legend_of_mana + dizzy + edl1 + edl2 + dungeon_kid + frontmission2 + rleinc1 + rleinc2 + evolution, aka lzf or lzfx + puyo_lz10 + puyo_lz11 + nislzs + unknown1 -> unknown19 + blackdesert + blackdesert_raw + pucrunch + zpaq + zyxel_lzs + blosc + gipfeli + crush + yappy + lzg + doboz + tornado + xpksqsh + amiga_unsquash + amiga_bytekiller + amiga_flashspeed + amiga_iamice + amiga_iamatm + amiga_isc1p + amiga_isc2p + amiga_isc3p + amiga_upcomp + amiga_uphd + amiga_bytekiller3 + amiga_bytekiller2 + amiga_crunchmania17b + amiga_powerpacker + amiga_stonecracker2 + amiga_stonecracker3 + amiga_stonecracker4 + amiga_crunchmaster + amiga_crunchmania + amiga_crunchmaniah + amiga_crunchomatic + amiga_discovery + amiga_lightpack + amiga_mastercruncher + amiga_maxpacker + amiga_megacruncher + amiga_packit + amiga_spikecruncher + amiga_tetrapack + amiga_timedecrunch + amiga_tryit + amiga_tuc + amiga_turbosqueezer61 + amiga_turbosqueezer80 + amiga_turtlesmasher + amiga_dms + amiga_packfire + alba_bpe + alba_bpe2 + flzp + sr2 + sr3 + bpe2v3 + bpe_alt1 + bpe_alt2 + cbpe + scpack0 + LZMA_0, headerless lzma, use the dictionary to + specify the dictionary size IF necessary + LZMA_86HEAD0 + LZMA_86DEC0 + LZMA_86DECHEAD0 + LZMA_EFS0 + LZMA2_0 + LZMA2_86HEAD0 + LZMA2_86DEC0 + LZMA2_86DECHEAD0 + LZMA2_EFS0 + lzovl + NITROSDK_DIFF8 + NITROSDK_DIFF16 + NITROSDK_HUFF8 + NITROSDK_HUFF16 + NITROSDK_LZ + NITROSDK_RL + qcmp + sparse + stormhuff + gzip_strict, gzip without autoguessing the presence + of crc32 and isize at the end, use this if the + file is a real 100% gzip file + CT_HughesTransform + CT_LZ77 + CT_ELSCoder + CT_RefPack + qfs, same as dk2 + PXP + BOH + GRC + ZEN + LZHUFXR + FSE + FSE_RLE + ZSTD, automatically supports all the legacy algos + CSC + RNCb + RNCb_RAW + RNCc_RAW + AZO + PP20 + DS_BLZ + DS_HUF + DS_LZE + DS_LZS + DS_LZX + DS_RLE + FAB + LZ4F + PCLZFG + LZOO + DELZC + DEHUFF + HEATSHRINK + NEPTUNIA + SMAZ + LZFX + PITHY + ZLING + DENSITY + BROTLI + RLE32 + RLE35 + BSC + SHOCO + WFLZ + FASTARI + RLE_ORCOM + DICKY + SQUISH + LZNT1 + XPRESS + XPRESS_HUFF + LZJODY + TRLE + SRLE + MRLE + JCH + LZRW1KH + LZSS0 + LHA_lz5 + LHA_lzs + LHA_lh1 + LHA_lh4 + LHA_lh5 + LHA_lh6 + LHA_lh7 + LHA_lhx + LHA_pm1 + LHA_pm2 + SQX1, currently it doesn't work at 100% + MDIP_ARAD + MDIP_ARST + MDIP_DELTA + MDIP_FREQ + MDIP_HUFFMAN + MDIP_CANONICAL + MDIP_LZSS + MDIP_LZW + MDIP_RICE + MDIP_RLE + MDIP_VPACKBITS + bizarre + bizarre_skip + lzssx + ash + YAY0 + DSTACKER + DSTACKER_SD3 + DSTACKER_SD4 + DBLSPACE + DBLSPACE_JM + XREFPACK, just another dk2/refpack + XREFPACK0, as above but without handling of header + qcmp2, UFG::qDecompressLZ probably the same of qcmp + deflatex, deflate used in Aeriagames pkg.idx, DICT + for btype order ("012") + zlibx, as above but skips the first 2 bytes + lzrw1a + lzrw2 + lzrw3a + lzrw5 + LEGO_IXS + mcomp, libmcomp / MCMQ / MCMP + mcomp0, rolz + mcomp1, rolz3 (or deflate fast on mcomp.exe) + mcomp2, lz + mcomp3, deflate + mcomp4, deflate64 + mcomp5, bzip2 + mcomp6, ppmdj + mcomp7, sl + mcomp8, sm + mcomp9, dmc + mcomp10, ??? (10,11,12,18,19,20 are all the same) + mcomp13, fpw1 + mcomp14, fpw2 + mcomp15, fpw3 + mcomp16, fpw4 + mcomp17, pwcm + irolz + irolz2 + uclpack + ace + ea_comp + ea_huff + ea_jdlz + tornado_byte + tornado_bit + tornado_huf + tornado_ari + lbalzss1 + lbalzss2 + dbpf, Maxis DBPF + TITUS_LZW + TITUS_HUFFMAN + KB_LZW + KB_DOSLZW + CARMACK + MBASH + DDAVE + GOT + SKYROADS + ZONE66 + EXEPACK + DE_LZW + JJRLE + K13RLE + SFRLC + WESTWOOD1 + WESTWOOD3 + WESTWOOD3b + WESTWOOD40 + WESTWOOD80 + PKWARE_DCL + TERSE + TERSE_SPACK_RAW + TERSE_PACK_RAW + REDUCE1 + REDUCE2 + REDUCE3 + REDUCE4 + LZW_ENGINE, requires the following parameters: + - initial codeword length (in bits) + - maximum codeword length (in bits) + - first valid codeword + - EOF codeword is first codeword + - reset codeword is shared with EOF + - flags (check src/compression/filter-lzw.h) + LZW_BASH + LZW_EPFS + LZW_STELLAR7 + ULTIMA6 + LZ5 + LZ5F + YALZ77 + LZKN1 + LZKN2 + LZKN3 + TFLZSS + SYNLZ1 + SYNLZ1b + SYNLZ1partial + SYNLZ2 + PPMZ2 + OPENDARK + DSLZSS + KOF, not working at 100% + KOF1, not working at 100% + RFPK + WP16 + LZ4_STREAM + OODLE + OODLE_LZH + OODLE_LZHLW + OODLE_LZNIB + OODLE_LZB16 + OODLE_LZBLW + OODLE_LZNA + OODLE_BitKnit + OODLE_LZA + OODLE_LZQ1, OODLE_kraken + OODLE_LZNIB2, OODLE_Mermaid + SEGS + OODLE_Selkie + OODLE_Akkorokamui + ALZ + REVELATION_ONLINE + ps_lz77 + lzfse + zle + KOF2, not working at 100% + KOF3, not working at 100% + HSQ + FACT5LZ + LZCAPTSU + TF3_RLE + WINIMPLODE + DZIP + DZIP_COMBUF, just a placeholder, doesn't work + LBALZSS1X, first version of LBALZSS1 (then fixed) + LBALZSS2X, first version of LBALZSS2 (then fixed) + GHIREN + FALCOM_DIN, automatically parses the chunks + FALCOM_DIN1, just input->output + FALCOM_DIN0, mode 0 + FALCOM_DINX, mode X + GLZA + M99CODER + LZ4X + TAIKO + LZ77EA_970 + DRV3_SRD + RECET + LIZARD, it's just the new name/version of LZ5 + MICROVISION + DR12AE + MSPACK, requires parameters + KONAMIAC + WOLF0, just WOLF without header + ARTSTATION + LEVEL5 + ZENPXP, dict must contain 0, 1, 2, 3, 4, 0xd, 0xe + or others supported + ZENPXP1, raw algo 1 of zenpxp + ZENPXP2, raw algo 2 of zenpxp + ZENPXP34, raw algo 3,4 of zenpxp + ZENPXPde, raw algo d,e of zenpxp + liblzs + SHREK + EA_MADDEN + nvcache, used by NVIDIA NV_Cache bin/toc + DE_HTML, same as the String command + HTML_EASY, same as the String command + JSON_VIEWER, same as the String command + XML_JSON_PARSER, same as the String command + OodleNetwork1UDP_State_Uncompact + OodleNetwork1_Shared_SetWindow, dict number of bits + OodleNetwork1UDP_Decode, requires the previous + usage of OodleNetwork1_Shared_SetWindow and + OodleNetwork1UDP_State_Uncompact + OodleNetwork1UDP_Encode + qcmp1, raw algorithm used in Sleeping Dogs + ykcmp + lzwab + ncompress, like uncompress plus the 3 bytes header + swzap + mzx + LZRRV + BCM + ULZ + SLZ_ROF, untested probably not working + LZ4X_NEW + COPY2 + SLZ_03b + MPPC + MPPC_BIG + alzss + clz + GTC + ANCO + ANCO0 + ANCO1 + ANCO2 + ANCO3 + ANCO4 + ANCO5 + konami_lz77 + vct_lzs + umesoft + systemaqua_catf + sogma + pac_ads + ail_lzs + agsi + foster_fa2 + an21 + arc_link + maika_bk + maika_mk2 + propeller_mgr + qlie + avg32_seen + sas5_iar + seraphim_scn + ugos_det + aaru_fl4 + inspire_ida + kurumi_mpk + dice_rlz + pulltop + vnsystem + QlzUnpack + umesoft_pk + tomcat_tcd + tail_pren + tail_crp0 + tail_hp + tactics_arc + sviu_pkz + nekox_gpc + rec_arc + warc + warc10 + warc_ylz + warc_huff + sh_him + pandora_pbx + origin_lz + origin_huffman + origin_rle + origin_alphav2 + garbro_huffman + ankh_grp + ankh_hdj + caramelbox_arc3 + caramelbox_arc4 + circus_V1 + circus_V2 + circus_V3 + cmvs_cpz + daisystem_pac + ethornell_bgi + fc01_mrg + fc01_mrg_quant + fc01_pak_lz + favorite_lzw + frontwing_rle + frontwing_huffman + g2_gcex + gss_arc + hypatia_mariel + interheart_fpk + kaguya_ari + kaguya_lin2 + kaguya_link + kaguya_uf + kid_dat + lambda_lax + microvision_arc + moonhir_fpk + spack + azsys + dxlib + glibg + gamesystem_cmp + puremail + groover_pcg + mnp_mma + strikes_pck + sega_lz77x + neptunia0, headerless chunk + puff8 + lzh8 + romchu + lzsd_of + lzsd_gfd + lzsd_gba2 + pzz + SL01 + rage_xfs + wangan1 + wangan2 + wangan3 + wangan5 + LZ48 + exo_decrunch + exo_decrunch_new + bitbuster + lazy + nibrans + LZRS_ASOBO + lzrhys + lze + zx0 + zx1 + zx2 + zx5 + rzip + melt1 + melt2 + + -------------------------------- + --- recompression algorithms --- + -------------------------------- + zlib_compress + deflate_compress + lzo1_compress + lzo1x_compress + lzo2a_compress + xmemlzx_compress + bzip2_compress + gzip_compress + lzss_compress + sfl_block_compress + sfl_rle_compress + sfl_nulls_compress + sfl_bits_compress + lzf_compress + brieflz_compress + jcalg_compress + bcl_huf_compress + bcl_lz_compress + bcl_rice_compress + bcl_rle_compress + bcl_sf_compress + szip_compress + huffmanlib_compress + lzma_compress + lzma_86head_compress + lzma_86dec_compress + lzma_86dechead_compress + lzma_efs_compress + falcom_compress + kzip_zlib_compress + kzip_deflate_compress + prs_compress + rnc_compress + lz4_compress + sfl_block_chunked_compress + snappy_compress + zpaq_compress + blosc_compress + gipfeli_compress + yappy_compress + lzg_compress + doboz_compress + nitrosdk_compress + hex_compress + base64_compress + lzma2_compress + lzma2_86head_compress + lzma2_86dec_compress + lzma2_86dechead_compress + lzma2_efs_compress + lzma_0_compress + lzma2_0_compress + stormhuff_compress + CT_HughesTransform_compress + CT_LZ77_compress + CT_ELSCoder_compress + CT_RefPack_compress + dk2_compress, also as ea_compress, refpack_compress + qfs_compress, same as dk2_compress + LZHUFXR_COMPRESS, aka STALKER_LZA_COMPRESS + FSE_COMPRESS + ZSTD_COMPRESS, only current algorithm, no legacy + DS_BLZ_COMPRESS + DS_HUF_COMPRESS + DS_LZE_COMPRESS + DS_LZS_COMPRESS + DS_LZX_COMPRESS + DS_RLE_COMPRESS + HEATSHRINK_COMPRESS + SMAZ_COMPRESS + LZFX_COMPRESS, DARKSECTOR_NOCHUNKS_COMPRESS, + FASTLZAH_COMPRESS, EVOLUTION_COMPRESS, + UNKNOWN6_COMPRESS + PITHY_COMPRESS + ZLING_COMPRESS + DENSITY_COMPRESS + BSC_COMPRESS + SHOCO_COMPRESS + WFLZ_COMPRESS + FASTARI_COMPRESS + DICKY_COMPRESS + SQUISH_COMPRESS + LZHL_COMPRESS + LZHAM_COMPRESS + TRLE_COMPRESS + SRLE_COMPRESS + MRLE_COMPRESS + CPK_COMPRESS, note that in reimport mode it may not + work because the decompression is size + dependent so the compressed size will probably + change but the value will not be modified in + the original archive (the reimport mode changes + only the data of the files). + if you want it to work you have to manually + edit the value from the archive, launch + quickbms -V and search something like: + . 00000eef get value 0x0000e548 4 + . 00000023 get column_name "FileSize" -1 + in that example you have to set the new value + at offset 0xeef, but note that the new + compressed size is not displayed by quickbms + LZRW1KH_COMPRESS + BPE_COMPRESS + NRV2b_COMPRESS + NRV2d_COMPRESS + NRV2e_COMPRESS + LZSS0_COMPRESS, aka PUYO_LZ01_COMPRESS + CLZW_COMPRESS + QUICKLZ_COMPRESS + ZOPFLI_ZLIB_COMPRESS + ZOPFLI_DEFLATE_COMPRESS + PKWARE_DCL_COMPRESS + LZ5_COMPRESS + YALZ77_COMPRESS + SYNLZ1_COMPRESS + SYNLZ2_COMPRESS + PPMZ2_COMPRESS + EA_JDLZ_COMPRESS + OODLE_COMPRESS + LZFSE_COMPRESS + M99CODER_COMPRESS + LZ4X_COMPRESS + YUKE_BPE_COMPRESS + LZO1A_COMPRESS + LZO1B_COMPRESS + LZO1C_COMPRESS + LZO1F_COMPRESS + LZO1Y_COMPRESS + LZO1Z_COMPRESS + LIZARD_COMPRESS + LIBLZS_COMPRESS + dizzy_compress + level5_compress + brotli_compress + DRV3_SRD_COMPRESS + YAZ0_COMPRESS + BIZARRE_COMPRESS + BIZARRE_SKIP_COMPRESS + BLACKDESERT_COMPRESS + DR12AE_COMPRESS + EA_COMP_COMPRESS + LBALZSS_COMPRESS + MOHLZSS_COMPRESS + MOHRLE_COMPRESS + NISLZS_COMPRESS + QCMP1_COMPRESS + RFPK_COMPRESS + RLEW_COMPRESS + SAINT_SEYA_COMPRESS + SHREK_COMPRESS + SLZ_01_COMPRESS + SLZ_02_COMPRESS + SLZ_03_COMPRESS + UCLPACK_COMPRESS + SEGA_LZS2_COMPRESS + WOLF_COMPRESS + YAKUZA_COMPRESS + YKCMP_COMPRESS + LZWAB_COMPRESS + NCOMPRESS_COMPRESS + UNCOMPRESS_COMPRESS + LZ4X_NEW_COMPRESS + DRAGONBALLZ_COMPRESS + MPPC_COMPRESS + MPPC_BIG_COMPRESS + ZLIBX_COMPRESS, just same as zlib_compress + DEFLATEX_COMPRESS, just same as deflate_compress + LZ77WII_COMPRESS, use dictionary for cmd, default + is 0x40 of DS_LZX_COMPRESS + APLIB_COMPRESS + LZ4F_COMPRESS + LZ5F_COMPRESS + ppmdh_compress + ppmdi_compress + lunar_lz1_compress to lunar_lz19_compress + lunar_rle1_compress to lunar_rle4_compress + coreonline_compress + + *note: the updated list is available in comtype.h + + DICT an optional C string containing the bytes of the + dictionary (cstring) or particular parameters + depending by the chosen algorithm. + Note that DICT can be: + - a cstring + comtype algo "\x11\x22\x33" // static binary dict + - a variable for which, often, you must specify + also a length + comtype algo DICT DICT_SIZE // variable dict \ No newline at end of file diff --git a/src/server/state.rs b/src/server/state.rs index 5565af6..8837ab7 100644 --- a/src/server/state.rs +++ b/src/server/state.rs @@ -472,6 +472,10 @@ pub fn get_keyword_docs() -> HashMap { "byte".to_string(), include_str!("keyword_docs/byte.txt").to_string(), ), + ( + "comtype".to_string(), + include_str!("keyword_docs/comtype.txt").to_string(), + ), ] .iter() .cloned() diff --git a/tree-sitter-quickbms/grammar.js b/tree-sitter-quickbms/grammar.js index ec5b2d4..fe7f184 100644 --- a/tree-sitter-quickbms/grammar.js +++ b/tree-sitter-quickbms/grammar.js @@ -45,6 +45,7 @@ module.exports = grammar({ $.append_statement, $.getvarchr_statement, $.putvarchr_statement, + $.comtype_statement, ), set_statement: $ => seq( $.set, @@ -243,6 +244,10 @@ module.exports = grammar({ field("type", optional($.type)), "\n", // Note: needed to avoid a conflict with a string statement following a getvarchr statement that does not have a type ), + comtype_statement: $ => seq( + $.comtype, + field("algorithm", $._expression), + ), comparison: $ => choice( "<", ">", @@ -363,6 +368,7 @@ module.exports = grammar({ append: $ => /[Aa][Pp][Pp][Ee][Nn][Dd]/, getvarchr: $ => /[Gg][Ee][Tt][Vv][Aa][Rr][Cc][Hh][Rr]/, putvarchr: $ => /[Pp][Uu][Tt][Vv][Aa][Rr][Cc][Hh][Rr]/, + comtype: $ => /[Cc][Oo][Mm][Tt][Yy][Pp][Ee]/, question_mark: $ => /\?/, identifier: $ => /[a-zA-Z_\\]+[a-zA-Z0-9_\-\\]*/, diff --git a/tree-sitter-quickbms/test/corpus/comtype.bms b/tree-sitter-quickbms/test/corpus/comtype.bms new file mode 100644 index 0000000..065a07f --- /dev/null +++ b/tree-sitter-quickbms/test/corpus/comtype.bms @@ -0,0 +1,12 @@ +================================================================================ +ComType statements +================================================================================ + +comtype COMP_UNZIP_DYNAMIC + +-------------------------------------------------------------------------------- + +(source_file + (comtype_statement + (comtype) + (identifier)))