Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using fifo_map #485

Closed
adah1972 opened this issue Mar 7, 2017 · 40 comments
Closed

Using fifo_map #485

adah1972 opened this issue Mar 7, 2017 · 40 comments
Labels
confirmed kind: bug solution: proposed fix a fix for the issue has been proposed and waits for confirmation

Comments

@adah1972
Copy link

adah1972 commented Mar 7, 2017

The document says "you can specialize the object type with containers like tsl::ordered_map or nlohmann::fifo_map". I had a try with simple code like:

#include "fifo_map.hpp"
#include "json.hpp"

using json = nlohmann::basic_json<nlohmann::fifo_map>;

int main()
{
    json j;
}

It fails the compilation noisily. Would you please fix the issue, and/or clarify in the document how this should be done?

@nlohmann
Copy link
Owner

nlohmann commented Mar 7, 2017

Can you please provide the compiler error messages?

@adah1972
Copy link
Author

adah1972 commented Mar 7, 2017

From GCC:

In file included from test.cpp:1:0:
fifo_map.hpp: In instantiation of 'nlohmann::fifo_map<Key, T, Compare, Allocator>::fifo_map() [with Key = std::__cxx11::basic_string<char>; T = nlohmann::basic_json<nlohmann::fifo_map>; Compare = std::less<std::__cxx11::basic_string<char> >; Allocator = std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::fifo_map> > >]':
/usr/local/Cellar/gcc/6.3.0_1/include/c++/6.3.0/ext/new_allocator.h:120:4:   required from 'void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = nlohmann::fifo_map<std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::fifo_map>, std::less<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::fifo_map> > > >; _Args = {}; _Tp = nlohmann::fifo_map<std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::fifo_map>, std::less<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::fifo_map> > > >]'
json.hpp:1634:9:   required from 'static T* nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::create(Args&& ...) [with T = nlohmann::fifo_map<std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::fifo_map>, std::less<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<nlohmann::fifo_map> > > >; Args = {}; ObjectType = nlohmann::fifo_map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long long int; NumberUnsignedType = long long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer'
json.hpp:1701:46:   required from 'nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::json_value::json_value(nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::value_t) [with ObjectType = nlohmann::fifo_map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long long int; NumberUnsignedType = long long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::value_t = nlohmann::detail::value_t]'
json.hpp:1917:49:   required from 'nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::basic_json(nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::value_t) [with ObjectType = nlohmann::fifo_map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long long int; NumberUnsignedType = long long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::value_t = nlohmann::detail::value_t]'
json.hpp:1941:35:   required from 'nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::basic_json(std::nullptr_t) [with ObjectType = nlohmann::fifo_map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long long int; NumberUnsignedType = long long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; std::nullptr_t = std::nullptr_t]'
test.cpp:8:10:   required from here
fifo_map.hpp:126:63: error: no matching function for call to 'std::less<std::__cxx11::basic_string<char> >::less(std::unordered_map<std::__cxx11::basic_string<char>, long unsigned int, std::hash<std::__cxx11::basic_string<char> >, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, long unsigned int> > >*)'
     fifo_map() : m_keys(), m_compare(&m_keys), m_map(m_compare) {}
                                                               ^

From Clang:

./fifo_map.hpp:126:28: fatal error: no matching constructor for initialization
      of 'std::__1::less<std::__1::basic_string<char> >'
    fifo_map() : m_keys(), m_compare(&m_keys), m_map(m_compare) {}
                           ^         ~~~~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/memory:1740:31: note: 
      in instantiation of member function
      'nlohmann::fifo_map<std::__1::basic_string<char>,
      nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator, adl_serializer>,
      std::__1::less<std::__1::basic_string<char> >,
      std::__1::allocator<std::__1::pair<const std::__1::basic_string<char>,
      nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator, adl_serializer> > > >::fifo_map' requested here
            ::new((void*)__p) _Up(_VSTD::forward<_Args>(__args)...);
                              ^
./json.hpp:1634:15: note: in instantiation of function template specialization
      'std::__1::allocator<nlohmann::fifo_map<std::__1::basic_string<char>,
      nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator, adl_serializer>,
      std::__1::less<std::__1::basic_string<char> >,
      std::__1::allocator<std::__1::pair<const std::__1::basic_string<char>,
      nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator, adl_serializer> > > >
      >::construct<nlohmann::fifo_map<std::__1::basic_string<char>,
      nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator, adl_serializer>,
      std::__1::less<std::__1::basic_string<char> >,
      std::__1::allocator<std::__1::pair<const std::__1::basic_string<char>,
      nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator, adl_serializer> > > >>' requested here
        alloc.construct(object.get(), std::forward<Args>(args)...);
              ^
./json.hpp:1701:30: note: in instantiation of function template specialization
      'nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator,
      adl_serializer>::create<nlohmann::fifo_map<std::__1::basic_string<char>,
      nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator, adl_serializer>,
      std::__1::less<std::__1::basic_string<char> >,
      std::__1::allocator<std::__1::pair<const std::__1::basic_string<char>,
      nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator, adl_serializer> > > >>' requested here
                    object = create<object_t>();
                             ^
./json.hpp:1917:31: note: in instantiation of member function
      'nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator, adl_serializer>::json_value::json_value' requested here
        : m_type(value_type), m_value(value_type)
                              ^
./json.hpp:1941:11: note: in instantiation of member function
      'nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator, adl_serializer>::basic_json' requested here
        : basic_json(value_t::null)
          ^
test.cpp:8:10: note: in instantiation of member function
      'nlohmann::basic_json<nlohmann::fifo_map, std::vector,
      std::__1::basic_string<char>, bool, long long, unsigned long long, double,
      std::allocator, adl_serializer>::basic_json' requested here
    json j;
         ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__functional_base:59:30: note: 
      candidate constructor (the implicit copy constructor) not viable: no known
      conversion from 'std::unordered_map<basic_string<char>, std::size_t> *'
      (aka 'unordered_map<std::__1::basic_string<char>, unsigned long> *') to
      'const std::__1::less<std::__1::basic_string<char> >' for 1st argument
struct _LIBCPP_TYPE_VIS_ONLY less : binary_function<_Tp, _Tp, bool>
                             ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__functional_base:59:30: note: 
      candidate constructor (the implicit move constructor) not viable: no known
      conversion from 'std::unordered_map<basic_string<char>, std::size_t> *'
      (aka 'unordered_map<std::__1::basic_string<char>, unsigned long> *') to
      'std::__1::less<std::__1::basic_string<char> >' for 1st argument
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__functional_base:59:30: note: 
      candidate constructor (the implicit default constructor) not viable:
      requires 0 arguments, but 1 was provided
1 error generated.

@nlohmann
Copy link
Owner

nlohmann commented Mar 7, 2017

Thanks, I shall have a look.

@nlohmann
Copy link
Owner

nlohmann commented Mar 7, 2017

I can confirm the error, but I have no idea how to fix it. I haven't tried fifo_map for this issue in a long time - I only mentioned it in the REAMDE, because I read somewhere that it was used successfully with this library. There may be issues, because fifo_map uses std::unordered_map internally, which is problematic (#164).

@nlohmann nlohmann added the state: help needed the issue needs help to proceed label Mar 7, 2017
@arvidsson
Copy link

Using Visual Studio C++ 2017, I get the following error:
'add_key': is not a member of 'std::less'

@Daniel599
Copy link
Contributor

Daniel599 commented Apr 10, 2017

Hi, I think I found out the problem, I think it's because when you are declaring object_t,
you use std::less, but not all map-like use std::less, i'm guessing it's because you wanted to set the allocator.
I can offer a patch:
Patch (might break people who set the map manually)
If you still want to keep you inner type allocator aware

template<class K, class V, class A>
using default_object_t = std::map<K, V, std::less< K >, A>; // A wrapper class for the new usage

template <
template<typename U, typename V, typename... Args> class ObjectType = default_object_t,
...
>
class basic_json
{
using object_t = ObjectType<StringType,
basic_json,
AllocatorType<std::pair<const StringType,
basic_json>>>; // Note I removed std::less
}

now if you want to use fifo_map you can by using the follow:

template<class K, class V, class A>
using my_obj = nlohmann::fifo_map<K, V, nlohmann::fifo_map_compare< K >, A>;

template<class K, class V, class A>
using my_obj2 = std::unordered_map<K, V, std::hash< K >, std::equal_to< K >, A>; // Can support also unordered_map

using my_json = nlohmann::basic_json<my_obj>;

please note that I haven't fully tested it, just basic example, not sure if it's the right way to do it, just an idea.

Also I can offer a workaround without changing the json library itself but it's a bit more ugly and will break if you will fix it using the patch above.

Let me know if you want this patch a pull request.

@donho
Copy link

donho commented Sep 10, 2017

Using Visual Studio C++ 2017, I get the following error:
'add_key': is not a member of 'std::less'

Same error under VS2015.
I think I will use nlohmann::json without fifo_map :(

@nlohmann
Copy link
Owner

Related to #660 (comment):

I really need to remove the hint to fifo_map from the documentation... :-(

@tomtachi
Copy link

tomtachi commented Sep 10, 2017 via email

@tomtachi
Copy link

tomtachi commented Sep 10, 2017 via email

@SiebelsTim
Copy link

The fifo_map library uses an uncommon comparator type, which needs to hold a key storage. std::map for example uses std::less by default, which is a simple functor without state.
The json library follows the pattern of std::map which directly passes std::less<StringType> to the object type. And of course, there is no constructor for std::less<std::string>(std::unordered_map<Key, std::size_t>*). That's the problem.

It is not possible to solve this without breaking compatibility with one of the libraries.

@Daniel599
Copy link
Contributor

Daniel599 commented Oct 2, 2017

As I said before in my previous comment, there is a way to do it without a change in the libraries, it's a workaround which I don't like that much since it will break if the API will change.
anyway here it's: (be warn - this workaround will break if the template parameters will change)

#include "json.hpp"
#include "fifo_map.hpp"
#include <iostream>

using namespace nlohmann;

// A workaround to give to use fifo_map as map, we are just ignoring the 'less' compare
template<class K, class V, class dummy_compare, class A>
using my_workaround_fifo_map = fifo_map<K, V, fifo_map_compare<K>, A>;
using my_json = basic_json<my_workaround_fifo_map>;

int main()
{
	my_json j;
	j["f"] = 5;
	j["a"] = 2;
	my_json j2 = {
	  {"pi", 3.141},
	  {"happy", true},
	  {"name", "Niels"},
	  {"nothing", nullptr},
	  {"answer", {
	    {"everything", 42}
	  }},
	  {"list", {1, 0, 2}},
	  {"object", {
	    {"currency", "USD"},
	    {"value", 42.99}
	  }}
	};

	std::cout << j << std::endl;
	std::cout << j2 << std::endl;

	return 0;
}

output:

{"f":5,"a":2}
{"pi":3.141,"happy":true,"name":"Niels","nothing":null,"answer":{"everything":42},"list":[1,0,2],"object":{"currency":"U
SD","value":42.99}}

I don't like this workaround very much since it isn't clean and really patchy, but maybe this will give others a better idea about what to do.

@SiebelsTim
Copy link

Yes, sure. I wasn't clear about that. I meant to say, it isn't possible if the goal is to allow basic_json<fifo_map>, which should be the goal in my opinion.

You are absolutely right. And I think it's worth thinking about documenting your work around in the Readme.

@nlohmann
Copy link
Owner

nlohmann commented Oct 5, 2017

@Daniel599 Thanks for sharing the workaround. I shall link it in the README until we find a better way.

@stale
Copy link

stale bot commented Nov 4, 2017

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the state: stale the issue has not been updated in a while and will be closed automatically soon unless it is updated label Nov 4, 2017
@lesf0
Copy link

lesf0 commented Nov 7, 2017

@Daniel599 's solution fails on this example:

#include "json.hpp"
#include "fifo_map.hpp"
#include <map>
#include <string>

using namespace nlohmann;

// A workaround to give to use fifo_map as map, we are just ignoring the 'less' compare
template<class K, class V, class dummy_compare, class A>
using my_workaround_fifo_map = fifo_map<K, V, fifo_map_compare<K>, A>;
using my_json = basic_json<my_workaround_fifo_map>;

int main()
{
	std::map<std::string, int> m = {{"one", 1}, {"two", 2}, {"three", 3}};
	my_json j = m;
}

I'm using GNU G++ compiler.

@stale stale bot removed the state: stale the issue has not been updated in a while and will be closed automatically soon unless it is updated label Nov 7, 2017
@nlohmann
Copy link
Owner

nlohmann commented Nov 7, 2017

Could you please provide the error message? Which version of GCC are you using?

@lesf0
Copy link

lesf0 commented Nov 7, 2017

% uname -a
Linux daddy 4.13.11-1-ARCH #1 SMP PREEMPT Thu Nov 2 10:25:56 CET 2017 x86_64 GNU/Linux

% g++ --version
g++ (GCC) 7.2.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

% g++ --std=c++17 file.cpp -o file
In file included from file.cpp:2:0:
fifo_map.hpp: In instantiation of ‘nlohmann::fifo_map<Key, T, Compare, Allocator>::fifo_map(InputIterator, InputIterator) [with InputIterator = std::_Rb_tree_const_iterator<std::pair<const std::__cxx11::basic_string<char>, int> >; Key = std::__cxx11::basic_string<char>; T = nlohmann::basic_json<my_workaround_fifo_map>; Compare = nlohmann::fifo_map_compare<std::__cxx11::basic_string<char> >; Allocator = std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<my_workaround_fifo_map> > >]’:
/usr/include/c++/7.2.0/ext/new_allocator.h:136:4:   required from ‘void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = nlohmann::fifo_map<std::__cxx11::basic_string<char>, nlohmann::basic_json<my_workaround_fifo_map>, nlohmann::fifo_map_compare<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<my_workaround_fifo_map> > > >; _Args = {std::_Rb_tree_const_iterator<std::pair<const std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >, std::_Rb_tree_const_iterator<std::pair<const std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >}; _Tp = nlohmann::fifo_map<std::__cxx11::basic_string<char>, nlohmann::basic_json<my_workaround_fifo_map>, nlohmann::fifo_map_compare<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<my_workaround_fifo_map> > > >]’
/usr/include/nlohmann/json.hpp:7932:9:   required from ‘static T* nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::create(Args&& ...) [with T = nlohmann::fifo_map<std::__cxx11::basic_string<char>, nlohmann::basic_json<my_workaround_fifo_map>, nlohmann::fifo_map_compare<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, nlohmann::basic_json<my_workaround_fifo_map> > > >; Args = {std::_Rb_tree_const_iterator<std::pair<const std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >, std::_Rb_tree_const_iterator<std::pair<const std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int> >}; ObjectType = my_workaround_fifo_map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer]’
/usr/include/nlohmann/json.hpp:784:26:   required from ‘static void nlohmann::detail::external_constructor<(nlohmann::detail::value_t)1>::construct(BasicJsonType&, const CompatibleObjectType&) [with BasicJsonType = nlohmann::basic_json<my_workaround_fifo_map>; CompatibleObjectType = std::map<std::__cxx11::basic_string<char>, int>; typename std::enable_if<(! std::is_same<CompatibleObjectType, typename BasicJsonType::object_t>::value), int>::type <anonymous> = 0]’
/usr/include/nlohmann/json.hpp:1038:53:   required from ‘void nlohmann::detail::to_json(BasicJsonType&, const CompatibleObjectType&) [with BasicJsonType = nlohmann::basic_json<my_workaround_fifo_map>; CompatibleObjectType = std::map<std::__cxx11::basic_string<char>, int>; typename std::enable_if<nlohmann::detail::is_compatible_object_type<BasicJsonType, CompatibleObjectType>::value, int>::type <anonymous> = 0]’
/usr/include/nlohmann/json.hpp:1337:23:   required from ‘decltype ((nlohmann::detail::to_json(j, forward<T>(val)), void())) nlohmann::detail::to_json_fn::call(BasicJsonType&, T&&, nlohmann::detail::priority_tag<1>) const [with BasicJsonType = nlohmann::basic_json<my_workaround_fifo_map>; T = std::map<std::__cxx11::basic_string<char>, int>&; decltype ((nlohmann::detail::to_json(j, forward<T>(val)), void())) = void]’
/usr/include/nlohmann/json.hpp:1352:64:   required from ‘void nlohmann::detail::to_json_fn::operator()(BasicJsonType&, T&&) const [with BasicJsonType = nlohmann::basic_json<my_workaround_fifo_map>; T = std::map<std::__cxx11::basic_string<char>, int>&]’
/usr/include/nlohmann/json.hpp:6877:28:   required from ‘static void nlohmann::adl_serializer< <template-parameter-1-1>, <template-parameter-1-2> >::to_json(BasicJsonType&, ValueType&&) [with BasicJsonType = nlohmann::basic_json<my_workaround_fifo_map>; ValueType = std::map<std::__cxx11::basic_string<char>, int>&; <template-parameter-1-1> = std::map<std::__cxx11::basic_string<char>, int>; <template-parameter-1-2> = void]’
/usr/include/nlohmann/json.hpp:8339:35:   required from ‘nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>::basic_json(CompatibleType&&) [with CompatibleType = std::map<std::__cxx11::basic_string<char>, int>&; U = std::map<std::__cxx11::basic_string<char>, int>; typename std::enable_if<((((! std::is_base_of<std::basic_istream<char>, U>::value) && (! std::is_same<U, nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer> >::value)) && (! nlohmann::detail::is_basic_json_nested_type<nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>, U>::value)) && nlohmann::detail::has_to_json<nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer>, U>::value), int>::type <anonymous> = 0; ObjectType = my_workaround_fifo_map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer]’
file.cpp:16:14:   required from here
fifo_map.hpp:131:55: error: no matching function for call to ‘nlohmann::fifo_map_compare<std::__cxx11::basic_string<char> >::fifo_map_compare(std::unordered_map<std::__cxx11::basic_string<char>, long unsigned int, std::hash<std::__cxx11::basic_string<char> >, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, long unsigned int> > >&)’
         : m_keys(), m_compare(m_keys), m_map(m_compare)
                                                       ^
fifo_map.hpp:51:5: note: candidate: nlohmann::fifo_map_compare<Key>::fifo_map_compare(std::unordered_map<Key, long unsigned int>*) [with Key = std::__cxx11::basic_string<char>]
     fifo_map_compare(std::unordered_map<Key, std::size_t>* k) : keys(k) {}
     ^~~~~~~~~~~~~~~~
fifo_map.hpp:51:5: note:   no known conversion for argument 1 from ‘std::unordered_map<std::__cxx11::basic_string<char>, long unsigned int, std::hash<std::__cxx11::basic_string<char> >, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, long unsigned int> > >’ to ‘std::unordered_map<std::__cxx11::basic_string<char>, long unsigned int, std::hash<std::__cxx11::basic_string<char> >, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, long unsigned int> > >*’
fifo_map.hpp:47:7: note: candidate: constexpr nlohmann::fifo_map_compare<std::__cxx11::basic_string<char> >::fifo_map_compare(const nlohmann::fifo_map_compare<std::__cxx11::basic_string<char> >&)
 class fifo_map_compare
       ^~~~~~~~~~~~~~~~
fifo_map.hpp:47:7: note:   no known conversion for argument 1 from ‘std::unordered_map<std::__cxx11::basic_string<char>, long unsigned int, std::hash<std::__cxx11::basic_string<char> >, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, long unsigned int> > >’ to ‘const nlohmann::fifo_map_compare<std::__cxx11::basic_string<char> >&’
fifo_map.hpp:47:7: note: candidate: constexpr nlohmann::fifo_map_compare<std::__cxx11::basic_string<char> >::fifo_map_compare(nlohmann::fifo_map_compare<std::__cxx11::basic_string<char> >&&)
fifo_map.hpp:47:7: note:   no known conversion for argument 1 from ‘std::unordered_map<std::__cxx11::basic_string<char>, long unsigned int, std::hash<std::__cxx11::basic_string<char> >, std::equal_to<std::__cxx11::basic_string<char> >, std::allocator<std::pair<const std::__cxx11::basic_string<char>, long unsigned int> > >’ to ‘nlohmann::fifo_map_compare<std::__cxx11::basic_string<char> >&&’

@Daniel599
Copy link
Contributor

I think I found the problem:
in fifo_map (line 134~)

    /// constructor for a range of elements
    template<class InputIterator>
    fifo_map(InputIterator first, InputIterator last)
        : m_keys(), m_compare(m_keys), m_map(m_compare)
    {
        for (auto it = first; it != last; ++it)
        {
            insert(*it);
        }
    }

Need to add & in m_compare(&m_keys)
fix:

    /// constructor for a range of elements
    template<class InputIterator>
    fifo_map(InputIterator first, InputIterator last)
        : m_keys(), m_compare(&m_keys), m_map(m_compare)
    {
        for (auto it = first; it != last; ++it)
        {
            insert(*it);
        }
    }

I hope it helps, need to make pull request to fifo_map I guess, maybe with some tests for this ctor.

@nlohmann
Copy link
Owner

nlohmann commented Nov 7, 2017

Thanks for looking into at. A PR would be greatly appreciated!

@nlohmann
Copy link
Owner

@Daniel599 Can you please check if the issue can be closed as nlohmann/fifo_map#4 is merged?

@Daniel599
Copy link
Contributor

after the merge, @zaycakitayca 's code now works.

@nlohmann
Copy link
Owner

Thanks a lot!

@nlohmann nlohmann added solution: proposed fix a fix for the issue has been proposed and waits for confirmation and removed state: help needed the issue needs help to proceed labels Nov 14, 2017
@DolphinDream
Copy link

If i define a fifo_map based json .. how can i have a method take as argument either a regular json or a fifo_map json value ?

template<class K, class V, class dummy_compare, class A>
using my_fifo_map = fifo_map<K, V, fifo_map_compare<K>, A>;
using my_json = basic_json<my_fifo_map>;

void processJson(json &j)
{
}

json j1;
my_json j2;

processJson(j1);
processJson(j2); // this does not compile 

@Daniel599
Copy link
Contributor

it doesn't compile because they are different types.
if you want same function for both of them, you need to convert processJson into a template function:

template<class JSONType>
void processJson(JSONType &j)
{
}
json j1;
my_json j2;

processJson(j1);
processJson(j2);

@risa2000
Copy link

I am sorry if this has been already solved, but I am trying to build specialized json with fifo_map, using following code:

#include <nlohmann/json.hpp>
#include <nlohmann/fifo_map.hpp>
// for convenience
using json = nlohmann::basic_json<nlohmann::fifo_map>;

and I am hit by the error already reported by @arvidsson above:

C:\PROGRA~2\MICROS~2\2017\COMMUN~1\VC\Tools\MSVC\1416~1.270\bin\HostX64\x64\cl.exe  /nologo /TP  -ID:\Dev\include\xtensor -ID:\Dev\include\nlohmann -ID:\Dev\include /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MDd /Zi /Ob0 /Od /RTC1   -std:c++17 /showIncludes /Fohmdq\CMakeFiles\hmdq.dir\hmdq.cpp.obj /Fdhmdq\CMakeFiles\hmdq.dir\ /FS -c ..\..\hmdq\hmdq.cpp
D:\Dev\include\nlohmann\fifo_map.hpp(188): error C2039: 'add_key': is not a member of 'std::less<void>'
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\include\xstddef(278): note: see declaration of 'std::less<void>'
  D:\Dev\include\nlohmann/fifo_map.hpp(187): note: while compiling class template member function 'T &nlohmann::fifo_map<StringType,T,nlohmann::basic_json<nlohmann::fifo_map,std::vector,StringType,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::object_comparator_t,std::allocator<std::pair<const StringType,nlohmann::basic_json<nlohmann::fifo_map,std::vector,StringType,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>>>>::operator [](const Key &)'
          with
          [
              T=nlohmann::basic_json<nlohmann::fifo_map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>,
              StringType=std::string,
              Key=std::string
          ]
  D:\Dev\include\nlohmann/json.hpp(3130): note: see reference to function template instantiation 'T &nlohmann::fifo_map<StringType,T,nlohmann::basic_json<nlohmann::fifo_map,std::vector,StringType,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::object_comparator_t,std::allocator<std::pair<const StringType,nlohmann::basic_json<nlohmann::fifo_map,std::vector,StringType,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>>>>::operator [](const Key &)' being compiled
          with
          [
              T=nlohmann::basic_json<nlohmann::fifo_map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>,
              StringType=std::string,
              Key=std::string
          ]
  D:\Dev\include\nlohmann/json.hpp(15909): note: see reference to class template instantiation 'nlohmann::fifo_map<StringType,nlohmann::basic_json<nlohmann::fifo_map,std::vector,StringType,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>,nlohmann::basic_json<nlohmann::fifo_map,std::vector,StringType,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>::object_comparator_t,std::allocator<std::pair<const StringType,nlohmann::basic_json<nlohmann::fifo_map,std::vector,StringType,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>>>>' being compiled
          with
          [
              StringType=std::string
          ]
  ..\..\hmdq\hmdq.cpp(50): note: see reference to class template instantiation 'nlohmann::basic_json<nlohmann::fifo_map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>' being compiled
  ninja: build stopped: subcommand failed.

I wonder, was it resolved, or is it a know bug? If the latter, is there any other solution?

@Daniel599
Copy link
Contributor

@risa2000 it doesn't seems like you used my workaround.
please define your json type as the example below.

As I said before in my previous comment, there is a way to do it without a change in the libraries, it's a workaround which I don't like that much since it will break if the API will change.
anyway here it's: (be warn - this workaround will break if the template parameters will change)

#include "json.hpp"
#include "fifo_map.hpp"
#include <iostream>

using namespace nlohmann;

// A workaround to give to use fifo_map as map, we are just ignoring the 'less' compare
template<class K, class V, class dummy_compare, class A>
using my_workaround_fifo_map = fifo_map<K, V, fifo_map_compare<K>, A>;
using my_json = basic_json<my_workaround_fifo_map>;

int main()
{
	my_json j;
	j["f"] = 5;
	j["a"] = 2;
	my_json j2 = {
	  {"pi", 3.141},
	  {"happy", true},
	  {"name", "Niels"},
	  {"nothing", nullptr},
	  {"answer", {
	    {"everything", 42}
	  }},
	  {"list", {1, 0, 2}},
	  {"object", {
	    {"currency", "USD"},
	    {"value", 42.99}
	  }}
	};

	std::cout << j << std::endl;
	std::cout << j2 << std::endl;

	return 0;
}

output:

{"f":5,"a":2}
{"pi":3.141,"happy":true,"name":"Niels","nothing":null,"answer":{"everything":42},"list":[1,0,2],"object":{"currency":"U
SD","value":42.99}}

I don't like this workaround very much since it isn't clean and really patchy, but maybe this will give others a better idea about what to do.

@risa2000
Copy link

@Daniel599 thanks for the nudge. Indeed I did not apply the "fix" as I did not understand that it was one! After following your example, it works as expected. Unfortunately, it turned out that I am stuck anyway.

The reason is that I am using xtensor library in the same project, which implements also a serialization/deserialization through nlohmann::json, but this library seems to be "hardcoding" nlohmann::json as the type it uses everywhere, with no template for possible specialization with a custom mapping class. So I guess I am going to write another report over there.

@theresa95
Copy link

theresa95 commented Apr 1, 2020

hey @nlohmann and @Daniel599 !

I am currently trying to generate JSON with ordered keys and therefore used the workaround method from @Daniel599 . I just copy-pasted your workaround code and it worked as aspected. However, if I try to use it within my base and derived classes, I get an error which I do not really understand. It seems like it fails to call the to_Json methods (because the error appears if I try to map a DerivedClass-instance (test and test2) to my_json.

I have already tried the example without ordered keys (just by using json = nlohmann::json;) and it works completely fine. The keys in the output are sorted alphabetically and looks like this:

{
  "cbor": "cbortest",
  "diagnostic": "diagnose: corona",
  "header": {
    "headerId": 3,
    "timestamp": "2019-12-10T16:04:00.00Z",
    "version": "4.0.0"
  },
  "hex": "00f2",
  "roundtrip": true
}

What I am trying to achieve through using the nlohmann fifo_map is to keep the insertion order and the final output therefore should look like this:

{
  "header": {
    "headerId": 3,
    "timestamp": "2019-12-10T16:04:00.00Z",
    "version": "4.0.0"
  },
  "cbor": "cbortest",
  "hex": "00f2",
  "roundtrip": true,
  "diagnostic": "diagnose: corona"
}

Executing the following code outputs two errors:

Error C2440: 'initializing': cannot convert from 'BaseNamespace::SubNamespace::DerivedClass' to 'nlohmann::basic_json<my_workaround_fifo_map,std::vector,std::string,bool,int64_t,uint64_t,double,std::allocator,nlohmann::adl_serializer>'	; in file: main.cpp

image

Please have a look at the following code:

In BaseClass.h:

#ifndef BASECLASS_H
#define BASECLASS_H

#include <stdint.h>
#include <string>
#include "nlohmann/json.hpp"
#include "fifo_map.hpp"

namespace BaseNamespace{

	namespace SubNamespace{

		class BaseClass {
		public:
			BaseClass () {};
			virtual ~BaseClass () {};

			uint32_t getHeaderId() const { return headerId; };
			std::string getTimestamp() const { return timestamp; };
			std::string getVersion() const { return version; };

			void setHeaderId(uint32_t str) { headerId = str; };
			void setTimestamp(std::string str) { timestamp = str; };
			void setVersion(std::string bo) { version = bo; };

			void setHeader(UAgvHeader const& header) {
				setHeaderId(header.getHeaderId());
				setTimestamp(header.getTimestamp());
				setVersion(header.getVersion());
			}

		private:
			uint32_t    headerId;       
			std::string timestamp;     
			std::string version;        
		};

		// A workaround to give to use fifo_map as map, we are just ignoring the 'less' compare
		using namespace nlohmann;
		template<class K, class V, class dummy_compare, class A>
		using my_workaround_fifo_map = fifo_map<K, V, fifo_map_compare<K>, A>;
		using my_json = basic_json<my_workaround_fifo_map>;

		void to_json(my_json &j, const BaseClass &p)
		{			
			j = my_json{
				{ "headerId", p.getHeaderId() }, 
				{ "timestamp", p.getTimestamp() }, 
				{ "version", p.getVersion() }
			};
		}

		void from_json(const my_json &j, BaseClass &p)
		{
			p.setHeaderId(j.at("headerId").get< std::uint32_t>());
			p.setTimestamp(j.at("timestamp").get< std::string >());
			p.setVersion(j.at("version").get<std::string>());
		}

	} // namespace SubNamespace

} // namespace BaseNamespace

#endif // BASECLASS_H_

In DerivedClass.h:

#ifndef DERIVEDCLASS_H
#define DERIVEDCLASS_H

#include <stdint.h>
#include <string>
#include "nlohmann/json.hpp"
#include <optional>
#include "BaseClass.h"

namespace BaseNamespace{

	namespace SubNamespace{

		class DerivedClass : public BaseClass {
		public:

			std::string getCBor() const { return cbor; };
			std::string getHex() const { return hex; };
			bool getRoundtrip() const { return roundtrip; };
			std::optional<std::string> getDiagnostic() const { return diagnostic; };

			void setCBor(std::string str) { cbor = str; };
			void setHex(std::string str) { hex = str; };
			void setRoundtrip(bool bo) { roundtrip = bo; };
			void setDiagnostic(std::optional<std::string> opt_str) { diagnostic = opt_str; };

		private:
			std::string cbor;
			std::string hex;
			bool        roundtrip;
			std::optional<std::string> diagnostic = std::nullopt;
		};
		
		void to_json(my_json &j, const DerivedClass& p)
		{
			j["header"] = static_cast<BaseClass>(p);
			j["cbor"] = p.getCBor();
			j["hex"] = p.getHex();
			j["roundtrip"] = p.getRoundtrip();

			// assuming you only want a "diagnostic" key if there is an actual value;
			// if not, store a nullptr and adjust the from_json accordingly
			if (p.getDiagnostic() != std::nullopt)
			{
				j["diagnostic"] = p.getDiagnostic().value();
			}
		}

		void from_json(const my_json &j, DerivedClass&p)
		{
			p.setHeader(j.at("header").get<BaseClass>());
			p.setCBor(j.at("cbor").get< std::string >());
			p.setHex(j.at("hex").get< std::string >());
			p.setRoundtrip(j.at("roundtrip").get< bool >());

			// if we also allow "null" values, then we need to add an "is_string()"
			// check
			if (j.count("diagnostic") != 0)
			{
				p.setDiagnostic(j.at("diagnostic").get< std::string >());
			}
		}

	} // namespace SubNamespace

} // namespace BaseNamespace

#endif // DERIVEDCLASS_H

In main.cpp:

#include <iostream>
#include <string>
#include <nlohmann/json.hpp>
#include <iomanip> 
#include <optional>
#include "DerivedClass.h"

using namespace nlohmann;
// A workaround to give to use fifo_map as map, we are just ignoring the 'less' compare
template<class K, class V, class dummy_compare, class A>
using my_workaround_fifo_map = fifo_map<K, V, fifo_map_compare<K>, A>;
using my_json = basic_json<my_workaround_fifo_map>;

int main(int argc, char* argv[]) {

	BaseNamespace::SubNamespace::DerivedClass test;
	test.setHeaderId(3);
	test.setTimestamp("2019-12-10T16:04:00.00Z");
	test.setVersion("4.0.0");
	test.setCBor("cbortest");
	test.setHex("00f2");
	test.setRoundtrip(true);
	test.setDiagnostic("diagnose: corona");
	my_json j = test;                                  // ERROR: no suitable conversion
	std::cout << std::setw(2) << j << std::endl;

	std::string str = R"({"header":
        {   "headerId" : 4711,
            "timestamp" : "1 Uhr",
		"version" : "5.0.0" 
        },
        "cbor" : "+X4A",
        "hex" : "f97e00",
        "roundtrip" : true,
        "diagnostic" : "NaN"
        })";

	my_json j2 = my_json::parse(str);
	BaseNamespace::SubNamespace::DerivedClass test2 = j2;
	my_json k = test2;                                 // ERROR: no suitable conversion

	std::cout << std::setw(2) << k << std::endl;


	return 0;
}

@Daniel599
Copy link
Contributor

Hi @theresa95, after some investigation I found out the cause of your error is due to multiple "my_json" types: one inside BaseNamespace::SubNamespace::my_json and the other in the global scope.
I don't know if it's the same case in your original code, but I mange to fix it by doing the following:

  • remove using namespace nlohmann; within your namespace (e.g inside BaseClass.h & DerivedClass.h)
  • define my_json in single place at BaseClass.h
    after doing the steps above, your code example compiles on g++ 7.5 with -std=c++17, I hope it compiles on windows as well (if not let me know)

@theresa95
Copy link

Hey @Daniel599 , first of all, thank you for the quick answer!

I edited the codesnippet of my question to fit your answer.
I removed the following lines from the DerivedClass.h file:

using namespace nlohmann;
template<class K, class V, class dummy_compare, class A>
using my_workaround_fifo_map = fifo_map<K, V, fifo_map_compare<K>, A>;
using my_json = basic_json<my_workaround_fifo_map>;

But I still get the same error messages in the main.cpp file:
image

Removing the "using namespace nlohmann" in the BaseClass.h file results in other errors so I have left it there:
image

What should I do with my_json definition and using namespace nlohmann in the main.cpp file?

Thank you so much for your help in advance!

@Daniel599
Copy link
Contributor

Daniel599 commented Apr 1, 2020

@theresa95
in BaseClass.h:

namespace BaseNamespace{
	namespace SubNamespace{
// Other code
		// A workaround to give to use fifo_map as map, we are just ignoring the 'less' compare
		template<class K, class V, class dummy_compare, class A>
		using my_workaround_fifo_map = nlohmann::fifo_map<K, V, nlohmann::fifo_map_compare<K>, A>;
		using my_json = nlohmann::basic_json<my_workaround_fifo_map>;
	}
}

note the addition of nlohmann:: which is needed when you remove the using namespace nlohmann

in main.cpp:

  • you can keep the using namespace nlohmann but I would recommend against it, prefer to use nlohmann::blabla whenever you need (P.S I don't think you need it in main anymore anyway)
  • replace my_json with BaseNamespace::SubNamespace::my_json

a bit more details:
since my_json is declared inside the namespace BaseNamespace::SubNamespace and main isn't (it's in the global scope), you have two options: add the namespace prefix or add using namespace BaseNamespace::SubNamespace in main.cpp, I like more the first option since it's more explicit about where things are taken from.

@theresa95
Copy link

@Daniel599 That did the trick! Thank you so much!

@ccpearson
Copy link

ccpearson commented Jul 29, 2021

Working with the latest nlohmann/json.hpp and nlohmann/fifo_map.hpp, I was also seeing the add_key() problem:

/home/ccpearson/src/.../include/nlohmann/fifo_map.hpp:188:19: error: ‘struct std::less<std::__cxx11::basic_string<char> >’ has no member named ‘add_key’
         m_compare.add_key(key);
         ~~~~~~~~~~^~~~~~~

This is how I'm using the library:

typedef nlohmann::basic_json<nlohmann::fifo_map> fifo_json;

I found that this one-line change to json.hpp fixed the compile problem, allowing the library to preserve the insertion order of object elements:

$ diff /usr/include/nlohmann/json.hpp ./include/nlohmann/json.hpp 
1280c1280
<           std::less<StringType>,
---
>           typename ObjectType<StringType, basic_json>::key_compare,

This was using nlohmann-json-dev 2.1.1-1.1 and the latest commit of fifo_map:

commit ece94577bc89759ff68244debb00e77f9c6de64e (HEAD -> master, origin/master, origin/HEAD)
Merge: fbcbb52 3a67b85
Author: Niels Lohmann <[email protected]>
Date:   Wed Mar 31 20:41:11 2021 +0200

    Merge pull request #12 from cbaecker/clang_cxxflags
    
    Fix CXX_FLAGS: "-stdlib=libc++" is clang-specific

With this change, the default (std::less) seems to work as before, but I haven't done any deep testing or tried this with other compilers, so this could be a works-for-me. FWIW...

@ccpearson
Copy link

Compile and system info for my previous comment:

$ g++ --version
g++ (Debian 8.3.0-6) 8.3.0

$ uname --all
Linux debian-vm 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux

HTH.

@nlohmann
Copy link
Owner

@ccpearson Thanks for sharing. I am confused - is fifo_map broken with the latest release? We use object_comparator_t now.

@ccpearson
Copy link

ccpearson commented Jul 30, 2021 via email

@ccpearson
Copy link

ccpearson commented Jul 30, 2021 via email

@ccpearson
Copy link

I'll just pull from GitHub and find out...

Yeah, there's no problem here, I'm just using an old version (2.1.1 from 2018). In the latest from github (3.9.1), nlohmann::ordered_json<> works for me. It looks like our team has standardized on an older debian version for some reason, possibly for h/w compatibility. Anyway, red herring.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
confirmed kind: bug solution: proposed fix a fix for the issue has been proposed and waits for confirmation
Projects
None yet
Development

No branches or pull requests