test failures on big endian.

[plugwash]

Automated testing in debian showed that the tests for the capstone crate failed on s390x (64-bit big endian)

https://ci.debian.net/data/autopkgtest/testing/s390x/r/rust-capstone/27661170/log.gz

failures:

---- test::test_arch_m68k_detail stdout ----
thread 'test::test_arch_m68k_detail' panicked at 'assertion failed: `(left == right)`
  left: `[M68kOperand(FpSingle(3.1415)), M68kOperand(Reg(RegId(17)))]`,
 right: `[M68kOperand(FpSingle(0.0)), M68kOperand(Reg(RegId(17)))]`: operands do not match for "0x1020: fadd.s #0.000000, fp0" (bytes=[f2, 3c, 44, 22, 40, 49, 0e, 56])', src/test.rs:254:5
stack backtrace:
   0: rust_begin_unwind
             at /usr/src/rustc-1.61.0/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:143:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:182:5
   4: capstone::test::test_instruction_detail_helper
             at ./src/test.rs:254:5
   5: capstone::test::instructions_match_detail
             at ./src/test.rs:445:9
   6: capstone::test::test_arch_mode_endian_insns_detail
             at ./src/test.rs:595:5
   7: capstone::test::test_arch_m68k_detail
             at ./src/test.rs:1781:5
   8: capstone::test::test_arch_m68k_detail::{{closure}}
             at ./src/test.rs:1760:1
   9: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.61.0/library/core/src/ops/function.rs:227:5
  10: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.61.0/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

---- test::test_arch_tms320c64x_detail stdout ----
thread 'test::test_arch_tms320c64x_detail' panicked at 'assertion failed: `(left == right)`
  left: `"NOP"`,
 right: `"nop"`: Did not match contained insn.mnemonic', src/test.rs:233:5
stack backtrace:
   0: rust_begin_unwind
             at /usr/src/rustc-1.61.0/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:143:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /usr/src/rustc-1.61.0/library/core/src/panicking.rs:182:5
   4: capstone::test::test_instruction_detail_helper
             at ./src/test.rs:233:5
   5: capstone::test::instructions_match_detail
             at ./src/test.rs:445:9
   6: capstone::test::test_arch_mode_endian_insns_detail
             at ./src/test.rs:595:5
   7: capstone::test::test_arch_tms320c64x_detail
             at ./src/test.rs:2471:5
   8: capstone::test::test_arch_tms320c64x_detail::{{closure}}
             at ./src/test.rs:2461:1
   9: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.61.0/library/core/src/ops/function.rs:227:5
  10: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.61.0/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

To determine whether these issues were still present in the latest git version of the capstone crate, whether these tests were specific to s390x or more general endian issues and whether 32-bit vs 64 made any difference I ran manual tests on powerpc (32-bit big endian) and ppc64 (64-bit big endian) with the latest version of capstone-rs from git.

The tests on ppc64 gave the same results as the automatic tests from Debian. The tests on powerpc gave a very different failure, a much larger number of tests failed, but they all failed with the same error.

---- arch::m68k::test::extra_info stdout ----
thread 'arch::m68k::test::extra_info' panicked at 'Failed to create Capstone: InvalidMode', capstone-rs/src/arch/m68k.rs:646:14

All tests were performed with rustc 1.61 from Debian.

[tmfink]

Thanks for the PR and analysis of the issue! I didn't even know this package was in the Debian repos.

Anyway, I'm having difficulty setting virtualizing an s390x or any other big ending platform with qemu-system. I had I noticed that the only big endian image available for download from debian is s390x. Do you have any instructions/links/recommendations on acquiring another archicture big endian image OR tips on emulating s390x?

I adapted some tips on running a Fedora s390x image from this blog (including pulling the kernel from the ISO):
https://people.redhat.com/~thuth/blog/qemu/2017/12/19/install-fedora.html

But I get a boot error:

$ qemu-system-s390x -m 1G -device virtio-scsi -drive file=debian-11.5.0-s390x-netinst.iso,format=raw,if=none,id=c1 -device scsi-cd,drive=c1 -hda disk.qcow2 --kernel linux_vm -nographic
KASLR disabled: CPU has no PRNG
[    1.613570] Linux version 5.10.0-18-s390x ([email protected]) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.140-1 (2022-09-02)
[    1.614873] setup: Linux is running under KVM in 64-bit mode
[    1.619440] setup: The maximum memory size is 1024MB
[    1.620595] cpu: 1 configured CPUs, 0 standby CPUs
[    1.642516] Write protected kernel read-only data: 10864k
[    1.657236] Zone ranges:
[    1.658653]   DMA      [mem 0x0000000000000000-0x000000007fffffff]
[    1.659124]   Normal   empty
[    1.659157] Movable zone start for each node
[    1.659188] Early memory node ranges
[    1.659266]   node   0: [mem 0x0000000000000000-0x000000003fffffff]
[    1.659492] Initmem setup node 0 [mem 0x0000000000000000-0x000000003fffffff]
[    1.669796] percpu: Embedded 34 pages/cpu s98560 r8192 d32512 u139264
[    1.671752] Built 1 zonelists, mobility grouping on.  Total pages: 258048
[    1.671902] Kernel command line: 
[    1.672683] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    1.672902] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    1.673593] mem auto-init: stack:off, heap alloc:on, heap free:off
[    1.678567] Memory: 244768K/1048576K available (9008K kernel code, 2052K rwdata, 1856K rodata, 3256K init, 604K bss, 37600K reserved, 0K cma-reserved)
[    1.684477] SLUB: HWalign=256, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    1.685253] ftrace: allocating 26440 entries in 104 pages
[    1.704542] ftrace: allocated 104 pages with 3 groups
[    1.710328] rcu: Hierarchical RCU implementation.
[    1.710366] rcu:     RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=1.
[    1.710457]  Rude variant of Tasks RCU enabled.
[    1.710467]  Tracing variant of Tasks RCU enabled.
[    1.710540] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    1.710569] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    1.731254] NR_IRQS: 3, nr_irqs: 3, preallocated irqs: 3
[    1.733965] clocksource: tod: mask: 0xffffffffffffffff max_cycles: 0x3b0a9be803b0a9, max_idle_ns: 1805497147909793 ns
[    1.737223] Console: colour dummy device 80x25
[    1.743683] printk: console [ttysclp0] enabled
[    1.744573] pid_max: default: 32768 minimum: 301
[    1.745975] LSM: Security Framework initializing
[    1.746988] Yama: disabled by default; enable with sysctl kernel.yama.*
[    1.748913] AppArmor: AppArmor initialized
[    1.748954] TOMOYO Linux initialized
[    1.749921] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    1.749953] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    1.782903] rcu: Hierarchical SRCU implementation.
[    1.789429] smp: Bringing up secondary CPUs ...
[    1.789530] smp: Brought up 1 node, 1 CPU
[    1.805803] node 0 deferred pages initialised in 12ms
[    1.816961] devtmpfs: initialized
[    1.824084] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    1.824274] futex hash table entries: 256 (order: 4, 65536 bytes, linear)
[    1.831837] NET: Registered protocol family 16
[    1.834185] audit: initializing netlink subsys (disabled)
[    1.837928] thermal_sys: Registered thermal governor 'fair_share'
[    1.838010] thermal_sys: Registered thermal governor 'bang_bang'
[    1.838034] thermal_sys: Registered thermal governor 'step_wise'
[    1.838042] thermal_sys: Registered thermal governor 'user_space'
[    1.838939] audit: type=2000 audit(1667806459.597:1): state=initialized audit_enabled=0 res=1
[    2.420762] iommu: Default domain type: Translated 
[    2.430722] NetLabel: Initializing
[    2.430769] NetLabel:  domain hash size = 128
[    2.430779] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    2.431489] NetLabel:  unlabeled traffic allowed by default
[    2.552755] VFS: Disk quotas dquot_6.6.0
[    2.552963] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    2.554068] hugetlbfs: disabling because there are no supported hugepage sizes
[    2.556505] AppArmor: AppArmor Filesystem Enabled
[    2.579631] NET: Registered protocol family 2
[    2.580692] IP idents hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    2.585781] tcp_listen_portaddr_hash hash table entries: 512 (order: 1, 8192 bytes, linear)
[    2.586031] TCP established hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    2.586300] TCP bind hash table entries: 8192 (order: 5, 131072 bytes, linear)
[    2.586619] TCP: Hash tables configured (established 8192 bind 8192)
[    2.587499] UDP hash table entries: 512 (order: 2, 16384 bytes, linear)
[    2.587703] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes, linear)
[    2.589108] NET: Registered protocol family 1
[    2.589323] NET: Registered protocol family 44
[    2.595958] hypfs: The hardware system does not support hypfs
[    2.597163] hypfs: Initialization of hypfs failed with rc=-61
[    2.600114] Initialise system trusted keyrings
[    2.601442] Key type blacklist registered
[    2.602362] workingset: timestamp_bits=46 max_order=18 bucket_order=0
[    2.612283] zbud: loaded
[    2.615586] integrity: Platform Keyring initialized
[    2.615922] Key type asymmetric registered
[    2.616025] Asymmetric key parser 'x509' registered
[    2.616299] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    2.617328] io scheduler mq-deadline registered
[    2.624466] hvc_iucv: The z/VM IUCV HVC device driver cannot be used without z/VM
[    2.627075] mousedev: PS/2 mouse device common for all mice
[    2.628378] ledtrig-cpu: registered to indicate activity on CPUs
[    2.629808] NET: Registered protocol family 10
[    2.661124] Segment Routing with IPv6
[    2.661510] mip6: Mobile IPv6
[    2.661635] NET: Registered protocol family 17
[    2.662435] mpls_gso: MPLS GSO support
[    2.663675] cio: Channel measurement facility initialized using format extended (mode autodetected)
[    2.665837] sclp_sd: Store Data request failed (eq=2, di=3, response=0x40f0, flags=0x00, status=0, rc=-5)
[    2.667824] ap: The hardware system does not support AP instructions
[    2.672277] registered taskstats version 1
[    2.672373] Loading compiled-in X.509 certificates
[    2.803691] Loaded X.509 cert 'Debian Secure Boot CA: 6ccece7e4c6c0d1f6149f3dd27dfcc5cbb419ea1'
[    2.804063] Loaded X.509 cert 'Debian Secure Boot Signer 2022 - linux: 14011249c2675ea8e5148542202005810584b25f'
[    2.804773] zswap: loaded using pool lzo/zbud
[    2.806796] Key type ._fscrypt registered
[    2.806839] Key type .fscrypt registered
[    2.806854] Key type fscrypt-provisioning registered
[    2.807993] AppArmor: AppArmor sha1 policy hashing enabled
[    2.813826] List of all partitions:
[    2.813934] No filesystem could mount root, tried: 
[    2.813961] 
[    2.814057] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(1,0)
[    2.814272] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.10.0-18-s390x #1 Debian 5.10.140-1
[    2.814291] Hardware name: QEMU 3906 QEMU (KVM/Linux)
[    2.814346] Call Trace:
[    2.814687]  [<00000000009b4912>] show_stack+0x8a/0xe0 
[    2.814751]  [<00000000009b91d8>] dump_stack+0x90/0xc0 
[    2.814763]  [<00000000009b5240>] panic+0x140/0x330 
[    2.814783]  [<0000000000d9fa02>] mount_block_root+0x40a/0x410 
[    2.814795]  [<0000000000d9fc0e>] prepare_namespace+0x166/0x1a0 
[    2.814806]  [<0000000000d9f3b0>] kernel_init_freeable+0x380/0x390 
[    2.814818]  [<00000000009bc7d0>] kernel_init+0x20/0x150 
[    2.814829]  [<00000000009c8dc4>] ret_from_fork+0x24/0x28 

[tmfink]

These types of failures are usually a bug in the upstream capstone C library.

Could you verify if the underlying C library shows the expected value? The cstool program can be used to test. Per my comment above, I don't have a big endian system to test on at the moment.

For example, on my amd64 Ubuntu machine, we can see the expected behavior from the test_arch_m68k_detail test case above:

Building cstool:

git clone -b next https://github.com/capstone-engine/capstone
cd capstone/
make -j12

Run test case:

./cstool/cstool m68k40 'f2 3c 44 22 40 49 0e 56'
 0  f2 3c 44 22 40 49 0e 56  fadd.s     #3.141500, fp0

[tmfink]

I was able to get a s390x virtualized using multiarch/qemu-user-static container as mentioned in these docs:
https://docs.gitlab.com/omnibus/development/s390x.html

It looks like the upstream C library has a bug when running on a big endian host:

$ uname -a
Linux d2dad0ba076b 5.19.0-76051900-generic #202207312230~1663791054~22.04~28340d4~dev-Ubuntu SMP PREEMPT_DY s390x s390x s390x GNU/Linux
$ ./cstool/cstool m68k40 'f2 3c 44 22 40 49 0e 56'
 0  f2 3c 44 22 40 49 0e 56  fadd.s     #0.000000, fp0