Introduction JSON module #379

Amaroq7 · 2016-07-14T23:50:06Z

Features

Supports decoding and encoding (also with pretty format)
Relies on Parson, which is lighweight and simple JSON library written in C
Supports dot notation (Values can be accessed by typing objectA.objectB.value)
Allows us to iterate through arrays and objects
Uses AMXXInterface but it is commented out for now until Add interface API #378 PR gets merged (for successful building)

How does it work?

Decoding

Sample JSON data was taken from GitHub API. We want to get information about last 2 commits (authors' names, their ids, committers' names and some extra).

#include <amxmodx>
#include <json>

public plugin_init()
{
    register_plugin("JSON decode example", "", "");
    register_srvcmd("test_json", "testJson");
}

public testJson()
{
    new JSON:root_value = json_parse("addons/amxmodx/data/json.txt", true);

    //Check if the root value is an array
    if (!json_is_array(root_value))
    {
        server_print("Something went wrong!");
        return;
    }

    server_print("File contains info about %d commits", json_array_get_count(root_value));  //Get elements in array

    new JSON:object;
    new data[200];  //array for stroring info

    //last 2 commits
    for (new i = 0; i < 2; i++)
    {
        object = json_array_get_value(root_value, i);

        json_object_get_string(object, "sha", data, charsmax(data));
        server_print("COMMIT %s", data);

        //Use dot notation
        json_object_get_string(object, "commit.author.name", data, charsmax(data), true);
        server_print("Author name: %s", data);
        server_print("Author id: %i", json_object_get_number(object, "author.id", true));
        server_print("Is author site admin: %s", json_object_get_bool(object, "author.site_admin", true) ? "YES" : "NO");

        json_object_get_string(object, "commit.committer.name", data, charsmax(data), true);
        server_print("Committer name: %s", data);
        server_print("Committer id: %i", json_object_get_number(object, "committer.id", true));
        server_print("Is committer site admin: %s", json_object_get_bool(object, "committer.site_admin", true) ? "YES" : "NO");

        server_print("/////////////////////////");
        json_free(object);
    }
    json_free(root_value);
}

Sample output

File contains info about 30 commits
COMMIT b9997eb628f975d72a8e1d5d556a4b4c0d1b6aec
Author name: IgnacioFDM
Author id: 12388603
Is author site admin: NO
Committer name: Vincent Herbet
Committer id: 360640
Is committer site admin: NO
/////////////////////////
COMMIT a53e7905db37f8e1b63929bdb27f8e5d0b10b1be
Author name: KliPPy
Author id: 9073673
Is author site admin: NO
Committer name: Vincent Herbet
Committer id: 360640
Is committer site admin: NO

Encoding

#include <amxmodx>
#include <json>

public plugin_init()
{
    register_plugin("JSON encode example", "", "");
    register_srvcmd("test_json", "testJson");
}

public testJson()
{
    //Create root value
    new JSON:root_value = json_init_object();

    //Set object
    json_object_set_string(root_value, "utf-8 string", "ęąćłóþπąœßćźżðæŋə©ś");
    json_object_set_string(root_value, "https", "https://alliedmods.net");
    json_object_set_string(root_value, "comment", "/**/");
    json_object_set_string(root_value, "text", "^"text^"");
    json_object_set_number(root_value, "number.int", 45, true);
    json_object_set_real(root_value, "number.float.easy", 31.7, true);
    json_object_set_real(root_value, "number.float.hard", -0.00132, true);
    json_object_set_null(root_value, "null");
    json_object_set_bool(root_value, "bool.true", true, true);
    json_object_set_bool(root_value, "bool.false", false, true);
    json_object_set_null(root_value, "//");

    //Create array
    new JSON:array = json_init_array();

    for (new i = 0; i < 5; i++)
    {
        json_array_append_number(array, i*i);
    }
    //Set array in object
    json_object_set_value(root_value, "x^^2 array", array);
    //Free array
    json_free(array);

    //Serialize to file with pretty format
    json_serial_to_file(root_value, "json_encode.txt", true);
    //Free our root value
    json_free(root_value);

    //Parse file
    root_value = json_parse("json_encode.txt", true);

    if(!json_is_object(root_value))
        return;

    new result[50];
    json_object_get_string(root_value, "utf-8 string", result, 49);
    server_print("UTF-8: %s", result);  //Get UTF-8 string
    json_object_get_string(root_value, "https", result, 49);
    server_print("HTTPS: %s", result);  //Get HTTPS address
    json_object_get_string(root_value, "text", result, 49);
    server_print("TEXT: %s", result);
    server_print("number.float.hard: %f", json_object_get_real(root_value, "number.float.hard", true)); //Get float value

    //Free object
    json_free(root_value);
}

Sample output in file

{
    "utf-8 string": "ęąćłóþπąœßćźżðæŋə©ś",
    "https": "https:\/\/alliedmods.net",
    "comment": "\/**\/",
    "text": "\"text\"",
    "number": {
        "int": 45,
        "float": {
            "easy": 31.700001,
            "hard": -0.001320
        }
    },
    "null": null,
    "bool": {
        "true": true,
        "false": false
    },
    "\/\/": null,
    "x^2 array": [
        0,
        1,
        4,
        9,
        16
    ]
}

Output in the console

UTF-8: ęąćłóþπąœßćźżðæŋə©ś
HTTPS: https://alliedmods.net
TEXT: "text"
Hard float: -0.001320

Iteration

We can iterate arrays and objects.
Arrays

//...
#include<json>
//...
public json_test()
{
        new JSON:value, JSON:array = json_parse("example.txt", true);
        new count = json_array_get_count(array);
        for (new i = 0; i < count; i++)
        {
                value = json_array_get_value(array, i);
                //Get type of value and do something
                //...
                //Free at the end
                json_free(value);
        }
        //Free array
        json_free(array);
}

Objects

//...
#include<json>
//...
public json_test()
{
        new JSON:value, JSON:object = json_parse("example.txt", true);
        new count = json_object_get_count(object);
        new key[20]; //stores key name
        for (new i = 0; i < count; i++)
        {
                json_object_get_name(object, i, key, charsmax(key));
                server_print("%s", key);
                value = json_object_get_value_at(object, i);
                //Get type of value and do something
                //...
                //Free at the end
                json_free(value);
        }
        //Free object
        json_free(object);
}

API Overview

json_parse parses JSON string or file that contains it.
json_equals checks equality of 2 values.
json_validate validates value.
json_get_parent gets value's parent.
json_get_type gets value type.
json_init_{type} creates handle with specific type.
json_get_{type} gets specific data from handle.
json_deep_copy copies value.
json_free frees handle.
Array API
json_array_get_{type} gets data from array.
json_array_get_count gets count of elements in array.
json_array_replace_{type} replaces value in array. (removes the old one if any)
json_array_append_{type} appends value in array.
json_array_remove removes value from specific index.
json_array_clear clears array.
Object API (supports dot notation)
json_object_get_{type} gets data from object.
json_object_get_count gets count of keys in object.
json_object_get_name gets name of object's key.
json_object_get_value_at gets value from object.
json_object_has_value checks if object has value and type.
json_object_set_{type} sets value in object. (removes the old one if any)
json_object_remove removes value in object.
json_object_clear clears object.
Serialization API
json_serial_size gets size of serialization.
json_serial_to_string copies serialized string to buffer.
json_serial_to_file copies serialized string to file.

Feel free to test it, give your thoughts and suggestions about code, documentation, API etc.

rsKliPPy · 2016-07-15T20:44:23Z

Do we really need JSON_Array and JSON_Object tags? Having just one tag for every node type feels better. Possibly JSON or JSON_Node, just to suggest a few possible names, although the former one seems more natural and is also shorter.

One thing I like in Jansson library is that it has macros to check node's type along with letting users manually check the type using a functions like json_value_get_type. Having something like:

json_is_array
json_is_object
json_is_string
json_is_number
json_is_null
json_is_bool[ean]

looks cleaner in my opinion. Just implement them as macros or short functions, not as native calls.

Also something that I never see people use, but is a really cool feature, is operator overloading for tags together with native function mapping. We could do that for json_value_equals here.

native bool:operator==(JSON_Value:value1, JSON_Value:value2) = json_value_equals;

although that eliminates to check if two handles ("references") are equal, but do we really need that in this context?

By the way, it's lovely that you decided to also support AMXXInterface. Great job overall. 👍

Amaroq7 · 2016-07-16T01:04:05Z

Do we really need JSON_Array and JSON_Object tags? Having just one tag for every node type feels better. Possibly JSON or JSON_Node, just to suggest a few possible names, although the former one seems more natural and is also shorter.

I did this because Parson only allows compare values (which can be arrays, objects etc.). If it was one tag f.e JSON: people may be confused. There are functions that returns arrays like json_array_get_array and values json_parse and think to compare them which will result an error. (tags are the same but first handle has only a pointer to array)

After some thinking I believe that can be done but there's need to rewrite some class functions f.e. in JSONMngr::ArrayGetArray we now have

int JSONMngr::ArrayGetArray(size_t array, size_t index)
{
    JSON_Array *JSArray = json_array_get_array(m_Handles[array]->m_pArray, index);

    return (JSArray) ? MakeHandle(JSArray, Handle_Array) : -1;
}

if we change it to

int JSONMngr::ArrayGetArray(size_t array, size_t index)
{
    JSON_Array *JSArray = m_Handles[array]->m_pArray;
    JSON_Array *JSChildArray = json_array_get_array(JSArray, index);

    if (!JSChildArray)
    {
        return -1;
    }

    //Check is above
    JSON_Value *JSValueChild = json_array_get_value(JSArray, index);

    size_t handleid = MakeHandle(JSChildArray, Handle_Array);
    SetHandleType(m_Handles[handleid], Handle_Value, reinterpret_cast<void *>(JSValueChild));

    return handleid;
}

we will be able to f.e. compare handles and with just one tag 😀. (because handles will have assigned value's pointer)
This must be also applied to JSONMngr::ArrayGetObject, JSONMngr::ObjectGetObject and JSONMngr::ObjectGetArray.

About macros, I think they would be useful and convenient.

About this:
native bool:operator==(JSON_Value:value1, JSON_Value:value2) = json_value_equals;
I didn't know it can be done this way until now 😄. Interesting feature.

although that eliminates to check if two handles ("references") are equal, but do we really need that in this context?

I don't understand what do you mean, maybe it's the time (it's around 3:00am) but it'd be great if you could explain me it in other words or more detailed.

IgnacioFDM · 2016-07-18T00:01:03Z

Awesome stuff

Amaroq7 · 2016-07-18T15:44:58Z

Update.

Removed:

json_value_get_{array/object} functions
json_{array/object}_get_{array/object} functions

For 1st there's no replacement because they aren't needed anymore.
For 2nd to get array or object we can use json_{array/object}_get_value.

Changed:

Instead of JSON_Value:, JSON_Array:, JSON_Object: now we have just JSON:.
We can compare values using == & != operators.
Renamed natives *_number_float to *_real.
Renamed native json_free_handle to json_free.
Merged json_object_has_value & json_object_has_value_type into one.
Removed value phrase from natives to make them shorter, f.e.

json_value_init_string -> json_init_string
json_value_deep_copy -> json_deep_copy and so on.

Added:

Macros for checking type

json_is_object,
json_is_array,
json_is_string,
json_is_number,
json_is_bool,
json_is_null,
json_is_true,
json_is_false.

Examples has been updated.

Arkshine · 2016-07-18T16:02:49Z

About the library licence, I think you need to add an entry here: https://github.com/alliedmodders/amxmodx/blob/master/public/licenses/ACKNOWLEDGEMENTS.txt.

Not a requirement, but I would not mind having a full pawn testsuite plugin.

In-line · 2016-07-28T06:47:02Z

Awesome

Arkshine · 2016-08-14T14:59:17Z

@Ni3znajomy are you alive?

Amaroq7 · 2016-08-14T15:44:21Z

@Arkshine Of course I am 😃 but I've been a little busy recently.

alldroll · 2016-09-06T12:29:02Z

@Arkshine why you don't want to use this?

Arkshine · 2016-09-06T15:00:21Z

It's not like I don't want. It just happened that Ni3znajomy did a PR related to JSON.
IMO, JSON/code overall seems better implemented in this PR.

About the library used, I don't know exactly what are the difference between Parson and Jansson. Looks like both are more or less the same. Here some benchmark. Parson seems easier though, and the dot notation is quite handy.

alldroll · 2016-09-06T15:53:55Z

@Arkshine but we have jansson wrapper for sourcemod too, so i thought it would be better to use same library. I can make code more better ( it will be nice if this module will be useful )

WPMGPRoSToTeMa · 2016-09-06T16:54:56Z

Some offtop:
Can we find appropriate use for this sort of macro?

#define SomeNamespace::%0.SomeFunction(%1) SomeNamespace_SomeFunction(%0, %1)

SomeNamespace::object.SomeFunction(arg);

For example:

if (JSON::value.IsObject())

Or:

JSON::value.GetString(buffer, charsmax(buffer));

Amaroq7 · 2016-09-06T23:18:12Z

@WPMGPRoSToTeMa Nice idea it looks more like SourcePawn syntax but I've got little problem with it, when there is 1 define it's ok, compiler starts complain when 2 or more defines appear then I got

AMX Mod X Compiler 1.8.3-dev+5070
Copyright (c) 1997-2006 ITB CompuPhase
Copyright (c) 2004-2013 AMX Mod X Team

test.sma(5) : warning 201: redefinition of constant/macro (symbol "JSON::%0.IsArray()")
test.sma(6) : warning 201: redefinition of constant/macro (symbol "JSON::%0.IsNum()")
Header size:            184 bytes
Code size:               24 bytes
Data size:               20 bytes
Stack/heap size:      16384 bytes
Total requirements:   16612 bytes

2 Warnings.
Done.

Plugin src

#include <amxmodx>
#include <json>

#define JSON::%0.IsObject() (%0 != Invalid_JSON && json_get_type(%0) == JSONObject)
#define JSON::%0.IsArray() (%0 != Invalid_JSON && json_get_type(%0) == JSONArray)
#define JSON::%0.IsNum() (%0 != Invalid_JSON && json_get_type(%0) == JSONNumber)

public plugin_init() { }

alldroll · 2016-09-07T00:41:49Z

@Ni3znajomy Maybe something like this

stock foo(JSON:handle, const method[])
{ 
// process it
}

#define JSON::%1.%2(); foo(%1, #%2)

public plugin_init()
{
   JSON::handle.IsObject()
}

/* or just declare enum */

enum MethodT {
  IsObject,
  IsString
};
stock foo(JSON:handle, MethodT:method)
{ 
// process it
}

#define JSON::%1.%2(); foo(%1, %2)

public plugin_init()
{
   JSON::handle.IsObject()
}

WPMGPRoSToTeMa · 2016-09-07T14:17:14Z

@Ni3znajomy

stock JSON_PrintSomething3(var, const str[], const str2[]) {
    server_print("%d %s %s", var, str, str2);
}
stock JSON_PrintSomething2(var, const str[]) {
    server_print("%d %s", var, str);
}
stock JSON_PrintSomething(var) {
    server_print("%d", var);
}

#define JSON::%0.%1(%2) JSON_%1(%0, %2)

    JSON::5.PrintSomething(); // This gives a compile error...
    JSON::5.PrintSomething2("a");
    JSON::5.PrintSomething3("a", "b");

But with this it works:

stock _JSON_PrintSomething(var) {
    server_print("%d", var);
}

#define JSON_PrintSomething(%0,%1) _JSON_PrintSomething(%0)

So, you can use this:

#define JSON_IsObject(%0,%2) (%0 != Invalid_JSON && json_get_type(%0) == JSONObject)
#define JSON_IsArray(%0,%2) (%0 != Invalid_JSON && json_get_type(%0) == JSONArray)
#define JSON_IsNum(%0,%2) (%0 != Invalid_JSON && json_get_type(%0) == JSONNumber)
#define JSON::%0.%1(%2) JSON_%1(%0, %2)

Arkshine · 2017-04-25T18:15:17Z

Reviewing takes time, especially when no one is willing to review. ¯\_(°╭╮°)_/¯
I can't tell when, but it's on the TODO list. ASAP.

justgo97 · 2017-05-09T18:15:13Z

ok hope it wont take long.

WPMGPRoSToTeMa · 2017-05-09T19:24:59Z

@Arkshine

¯_(°╭╮°)_/¯

¯\_(°╭╮°)_/¯

Fix memory leaks and increase max nesting

Order of items in an array is preserved after removing an item.

Arkshine · 2017-09-25T16:07:01Z

If possible, can you provide a VS project?

Amaroq7 · 2017-09-27T17:46:20Z

I don't use nor have VS 😕

Arkshine · 2017-09-27T19:24:44Z

It's okay, I will PR your branch later.

By the way, did you test your module after all these late changes?
I'm saying that because I wanted to try your test plugin (first command) and it crashed HLDS (windows).
I did not investigate yet, just wanted to share this behavior.

Amaroq7 · 2017-09-27T20:11:48Z

Yeah, there was a mistake when I started using ke::AutoPtrs to manage JSONHandles, forgot that I have to create objects manually. Should be ok now.

Add VS projects files and fix mixed tab/spaces

Arkshine · 2017-09-27T22:18:19Z

modules/json/JsonNatives.cpp

+	auto value = MF_GetAmxAddr(amx, params[1]);
+	if (!JsonMngr->IsValidHandle(*value))
+	{
+		MF_LogError(amx, AMX_ERR_NATIVE, "Invalid JSON value! %d", *value);


I'm not sure erroring on json_free is useful. I believe others APIs don't do that as well.

Gonna be removed.

Done along with comments.

Arkshine · 2017-09-27T22:19:54Z

plugins/include/json.inc

+	#pragma loadlib json
+#endif
+
+enum JSONType


Missing comment.

Arkshine · 2017-09-27T22:20:45Z

plugins/include/json.inc

+    JSONBoolean = 6
+};
+
+enum JSON


Missing comment.

Arkshine · 2017-09-28T18:36:17Z

It looks like in a good & sane place now. The code is quite straightforward.
I don't know if it misses features in the pawn API, but it seems complete and simple to use. At some point, we need people's feedback anyway and new features can be requested later.

If someone wants to say something about this module, let's hear now, before merging happens.

rsKliPPy · 2017-09-29T18:07:55Z

plugins/include/json.inc

+/*
+ * Overloaded operators for convenient comparing
+ */
+native bool:operator==(const JSON:value1, const JSON:value2) = json_equals;


I know I initially suggested this, but after putting more thought into it, I'd actually get rid of it.
While it's a cool feature, no other handle types in AMXX override operator==. People are used to operator== comparing handles/references, not their data, so an implicit deep comparison of JSON objects may be a bad idea.

I agree as well, deep comparison should be explicit.

rsKliPPy · 2017-09-29T18:12:05Z

modules/json/JsonMngr.h

+	virtual ~JSONMngr();
+
+	// Handles
+	virtual bool IsValidHandle(JS_Handle id, JSONHandleType type = Handle_Value);


You should use override here and with any other overridden method instead of virtual. So instead of

virtual bool IsValidHandle(JS_Handle id, JSONHandleType type = Handle_Value);

it should be

bool IsValidHandle(JS_Handle id, JSONHandleType type = Handle_Value) override;

Here virtual indicates that given method may be overridden further by a derived class. override ensures that the overridden method exists in the base class (IJSONMngr) and is of the same type.

Arkshine · 2017-09-30T18:22:40Z

Alright. Let's do it!

Amaroq7 force-pushed the json-module branch 8 times, most recently from 563b4dd to c85d621 Compare July 15, 2016 15:27

Amaroq7 force-pushed the json-module branch 6 times, most recently from 6547166 to 1cd76a3 Compare July 18, 2016 15:29

Amaroq7 force-pushed the json-module branch from 1cd76a3 to 492bf05 Compare July 18, 2016 15:53

Amaroq7 added 5 commits September 4, 2017 21:20

Merge branch 'master' into json-module

162373e

Merge upstream changes

730349e

Fix memory leaks and increase max nesting

Fix documentation

56d86c9

Use AutoPtr in managing JSON handles

1acc7cc

Merge upstream changes

cf895be

Order of items in an array is preserved after removing an item.

Merge upstream

9e18f84

Fix crash

cffaaa0

Arkshine and others added 2 commits September 27, 2017 23:52

Add VS projects files and fix mixed tab/spaces

2a44a89

Merge pull request #1 from Arkshine/json-module

6bcf642

Add VS projects files and fix mixed tab/spaces

Arkshine reviewed Sep 27, 2017

View reviewed changes

Arkshine added the module: json label Sep 27, 2017

Amaroq7 added 2 commits September 28, 2017 00:40

Remove erroring on "json_free"

8b52647

Add comments to "json.inc" file

ccf06a6

rsKliPPy suggested changes Sep 29, 2017

View reviewed changes

Amaroq7 added 3 commits September 30, 2017 19:13

Remove overloaded operators

92c4ba1

Use of "override" keyword where needed

c4f8723

Fix parameter's name

0406a24

Arkshine merged commit 361a6cc into alliedmodders:master Sep 30, 2017

Amaroq7 deleted the json-module branch September 30, 2017 20:24

CrazyHackGUT mentioned this pull request Apr 7, 2021

Built-in JSON module alliedmodders/sourcemod#1468

Closed

ProjectSky mentioned this pull request Oct 18, 2025

Introduction JSON extension alliedmodders/sourcemod#2365

Draft

Uh oh!

Introduction JSON module #379

Introduction JSON module #379

Uh oh!

Conversation

Amaroq7 commented Jul 14, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Features

How does it work?

Decoding

Encoding

Iteration

API Overview

Uh oh!

rsKliPPy commented Jul 15, 2016

Uh oh!

Amaroq7 commented Jul 16, 2016

Uh oh!

IgnacioFDM commented Jul 18, 2016

Uh oh!

Amaroq7 commented Jul 18, 2016

Uh oh!

Arkshine commented Jul 18, 2016

Uh oh!

In-line commented Jul 28, 2016

Uh oh!

Arkshine commented Aug 14, 2016

Uh oh!

Amaroq7 commented Aug 14, 2016

Uh oh!

alldroll commented Sep 6, 2016

Uh oh!

Arkshine commented Sep 6, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alldroll commented Sep 6, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WPMGPRoSToTeMa commented Sep 6, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Amaroq7 commented Sep 6, 2016

Uh oh!

alldroll commented Sep 7, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WPMGPRoSToTeMa commented Sep 7, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Arkshine commented Apr 25, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

justgo97 commented May 9, 2017

Uh oh!

WPMGPRoSToTeMa commented May 9, 2017

Uh oh!

Arkshine commented Sep 25, 2017

Uh oh!

Amaroq7 commented Sep 27, 2017

Uh oh!

Arkshine commented Sep 27, 2017

Uh oh!

Amaroq7 commented Sep 27, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Arkshine commented Sep 28, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Amaroq7 commented Jul 14, 2016 •

edited

Loading

Arkshine commented Sep 6, 2016 •

edited

Loading

alldroll commented Sep 6, 2016 •

edited

Loading

WPMGPRoSToTeMa commented Sep 6, 2016 •

edited

Loading

alldroll commented Sep 7, 2016 •

edited

Loading

WPMGPRoSToTeMa commented Sep 7, 2016 •

edited

Loading

Arkshine commented Apr 25, 2017 •

edited

Loading

Amaroq7 commented Sep 27, 2017 •

edited

Loading