See #1275 where these were removed. The upstream spec was changed to use a more generic syntax:
(assert_return_canonical_nan (invoke "..."))
becomes
(assert_return (invoke "...") (nan:canonical))
(And similarly for assert_return_arithmetic_nan and nan:arithmetic).
But in general, wabt tries to be backward compatible, so it could continue to support this old syntax too.
See WebAssembly/bulk-memory-operations#144.