ADM (Audio Definition Model) and Dolby Metadata in WAV/RIFF

README.md

@@ -69,6 +69,7 @@ bzip2,
 [csv](doc/formats.md#csv),
 dns,
 dns_tcp,
+[dolby_metadata](doc/formats.md#dolby_metadata),
 elf,
 ether8023_frame,
 exif,
@@ -157,7 +158,7 @@ vp9_cfm,
 vp9_frame,
 vpx_ccr,
 [wasm](doc/formats.md#wasm),
-wav,
+[wav](doc/formats.md#wav),
 webp,
 [xml](doc/formats.md#xml),
 yaml,

doc/dev.md

@@ -38,16 +38,27 @@ Flags can be struct with bit-fields.
   - Use commit messages with a context prefix to make it easier to find and understand, ex:<br>
   `mp3: Validate sync correctly`
 - Tests:
-  - If possible use a pair of `testdata/file` and `testdata/file.fqtest` where `file.fqtest` is `$ fq dv file` or `$ fq 'dv,torepr' file` if there is `torepr` support.
+  - If possible, add one or more pairs of example input binary file plus an `.fqtest` file with one or more expected CLI commands and their outputs, with naming like:
+    - `./format/<format_name>/testdata/<format_ex_file_name>.<ext>`, e.g. [`./format/mp4/testdata/aac.mp4`](../format/mp4/testdata/aac.mp4)
+    - and `./format/<format_name>/testdata/<format_ex_file_name>.fqtest`, e.g. [`./format/mp4/testdata/aac.fqtest`](../format/mp4/testdata/aac.fqtest)
+  - The `*.fqtest` files use lines prefixed with `$` of commands to run / test (usually `fq` command style of course), and their expected output below them, e.g. in simple [`./pkg/interp/testdata/version.fqtest`](../pkg/interp/testdata/version.fqtest):
+    ```
+    $ fq -v
+    testversion (testos testarch)
+    ```
+  - A basic test of a (potentially new) format can be specified by creating a single line `./format/<format_name>/testdata/<format_ex_file_name>.fqtest` with `$ fq dv <format_ex_file_name>.<ext>` or `$ fq 'dv,torepr' <format_ex_file_name>.<ext>` if there is `torepr` support. (Note: for the test file, do not use full path like `./format/<format_name>/testdata/<format_ex_file_name>.<ext>`)
+  - You can of course look at your code's current output for a test like this by locally running the same command using `go` and with full path, e.g. `go run . dv ./format/<format_name>/testdata/<format_ex_file_name>.<ext>`
   - If `dv` produces a lof of output maybe use `dv({array_truncate: 50})` etc
-  - Run `go test ./format -run TestFormats/<name>` to test expected output.
-  - Run `go test ./format -run TestFormats/<name> -update` to update current output as expected output.
+  - Run `go test ./format -run TestFormats/<format_name>` to test expected outputs for all tests under format `<format_name>`.
+  - Run `go test ./format -run TestFormats/<format_name> -update` to automatically update current (or create, on initial addition of new test files and lines) expected output with the currently locally executing content.
+  - If you've added a new format, you'll probably also see that the generic `all` test fails, update that as well with: `go test ./format -run TestFormats/all -update`
+  - Double check that your git diff looks sensible for new or updated `*.fqtest` files (e.g. avoiding checking in test failure output like `exitcode: 2 error: wrong_path.mp4: no such file or directory`)
 - If you have format specific documentation:
   - Put it in `format/*/<name>.md` and use `//go:embed <name>.md`/`interp.RegisterFS(..)` to embed/register it.
   - Use simple markdown, just sections (depth starts at 3, `### Section`), paragraphs, lists and links.
-  - No heading section is needs with format name, will be added by `make doc` and fq cli help system.
+  - No heading section is needed with format name, will be added by `make doc` and fq cli help system.
   - Add a `testdata/<name>_help.fqtest` with just `$ fq -h <name>` to test CLI help.
-  - If in doubt look at `mp4.md`/`mp4.go` etc.
+  - If in doubt look at [`mp4.md`](../format/mp4/mp4.md)/[`mp4.go`](../format/mp4/mp4.go) etc.
   - Run `make README.md doc/formats.md` to update md files.
 - Run linter `make lint`
 - Run fuzzer `make fuzz GROUP=<name>`, see usage in Makefile

doc/formats.md

@@ -41,6 +41,7 @@
 |[`csv`](#csv)                                                   |Comma&nbsp;separated&nbsp;values                                                                             |<sub></sub>|
 |`dns`                                                           |DNS&nbsp;packet                                                                                              |<sub></sub>|
 |`dns_tcp`                                                       |DNS&nbsp;packet&nbsp;(TCP)                                                                                   |<sub></sub>|
+|[`dolby_metadata`](#dolby_metadata)                             |Dolby&nbsp;Metadata&nbsp;(Atmos,&nbsp;AC3,&nbsp;Dolby&nbsp;Digital)                                          |<sub></sub>|
 |`elf`                                                           |Executable&nbsp;and&nbsp;Linkable&nbsp;Format                                                                |<sub></sub>|
 |`ether8023_frame`                                               |Ethernet&nbsp;802.3&nbsp;frame                                                                               |<sub>`inet_packet`</sub>|
 |`exif`                                                          |Exchangeable&nbsp;Image&nbsp;File&nbsp;Format                                                                |<sub></sub>|
@@ -129,7 +130,7 @@
 |`vp9_frame`                                                     |VP9&nbsp;frame                                                                                               |<sub></sub>|
 |`vpx_ccr`                                                       |VPX&nbsp;Codec&nbsp;Configuration&nbsp;Record                                                                |<sub></sub>|
 |[`wasm`](#wasm)                                                 |WebAssembly&nbsp;Binary&nbsp;Format                                                                          |<sub></sub>|
-|`wav`                                                           |WAV&nbsp;file                                                                                                |<sub>`id3v2` `id3v1` `id3v11`</sub>|
+|[`wav`](#wav)                                                   |WAV&nbsp;file                                                                                                |<sub>`id3v2` `id3v1` `id3v11` `dolby_metadata`</sub>|
 |`webp`                                                          |WebP&nbsp;image                                                                                              |<sub>`exif` `vp8_frame` `icc_profile` `xml`</sub>|
 |[`xml`](#xml)                                                   |Extensible&nbsp;Markup&nbsp;Language                                                                         |<sub></sub>|
 |`yaml`                                                          |YAML&nbsp;Ain't&nbsp;Markup&nbsp;Language                                                                    |<sub></sub>|
@@ -590,6 +591,33 @@ $ fq -d csv -o comma="\t" to_csv file.tsv
 $ fq -d csv '.[0] as $t | .[1:] | map(with_entries(.key = $t[.key]))' file.csv
 ```
 
+## dolby_metadata
+Dolby Metadata (Atmos, AC3, Dolby Digital).
+
+Dolby Metadata from `<dbmd>` chunk of RIFF / WAV / Broadcast Wave Format (BWF),
+including Dolby Atmos, AC3, Dolby Digital \[Plus\], and Dolby Audio Info (e.g. LUFS, True Peak).
+
+### Examples
+Decode Dolby metadata from `<dbmd>` chunk:
+```
+$ fq -d wav '.chunks[] | select(.id | IN("dbmd")) | tovalue' adm-bwf.wav
+```
+
+RIFF / WAV / Broadcast Wave Format (BWF) chunks:
+- `<chna>` Track UIDs of Audio Definition Model
+- `<axml>` BWF XML Metadata, e.g. for Audio Definition Model ambisonics and elements
+
+### Authors
+- [@johnnymarnell](https://johnnymarnell.github.io), original author
+
+### References
+- https://adm.ebu.io/background/what_is_the_adm.html
+- https://tech.ebu.ch/publications/tech3285s7
+- https://tech.ebu.ch/publications/tech3285s5
+- https://tech.ebu.ch/files/live/sites/tech/files/shared/tech/tech3285s6.pdf
+- https://github.com/DolbyLaboratories/dbmd-atmos-parser
+- https://github.com/MediaArea/MediaInfoLib/blob/master/Source/MediaInfo/Audio/File_DolbyAudioMetadata.cpp
+
 ## fit
 Garmin Flexible and Interoperable Data Transfer.
 
@@ -1442,6 +1470,49 @@ $ fq '.sections | {import: map(select(.id == "import_section").content.im.x[].nm
 ### References
 - https://webassembly.github.io/spec/core/
 
+## wav
+WAV file.
+
+WAVE audio file format.
+
+Also includes support for [Audio Definition Model](https://adm.ebu.io/background/what_is_the_adm.html) and 3D Audio.
+
+RIFF / WAV / Broadcast Wave Format (BWF) chunks:
+
+- `RIFF`: primary container chunk specifying the file type and containing sub-chunks (e.g., fmt, data)
+- `fmt`: describes format / stream encoding in data chunk
+- `data`: indicates size and contains encoded raw sound data
+- `bext`: broadcast extension chunk, containing broadcast-specific metadata such as description, originator, creation date, time reference, and more
+- `LIST`: organizes additional metadata in sub-chunks, often used to include information like artist, genre, or title in INFO or other standardized formats
+- `smpl`: sample metadata chunk, containing looping and sampling information, such as start and end points for loops, sample rate, and MIDI pitch
+- `fact`: contains metadata on the original uncompressed data, such as the number of samples, typically used in non-PCM (compressed) formats to aid in playback and synchronization
+- `chna`: track UIDs of Audio Definition Model
+- `axml`: XML metadata, e.g. for Audio Definition Model ambisonics and elements as in [EBUCore spec](https://tech.ebu.ch/docs/tech/tech3293.pdf)
+- `dbmd`: Dolby specific metadata like loudness and binaural settings, see also [`dolby_metadata` format](#dolby_metadata)
+
+
+### Examples
+Decode ADM configuration from `<chna>` and `<axml>` chunks:
+```bash
+$ fq -d wav '.chunks[] | select(.id | IN("chna", "axml")) | tovalue' amd-bwf.wav
+
+# Extract ADM <axml> chunk objects definitions xml content
+$ fq -r -d wav '.chunks[] | select(.id | IN("axml")) | .xml | tovalue' amd-bwf.wav | tee axml-content.xml
+```
+
+### Authors
+- [@wader](https://github.com/wader), original author
+- [@johnnymarnell](https://johnnymarnell.github.io), ADM support
+
+### References
+- http://soundfile.sapp.org/doc/WaveFormat/
+- https://github.com/FFmpeg/FFmpeg/blob/master/libavformat/wavdec.c
+- https://tech.ebu.ch/docs/tech/tech3285.pdf
+- http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/WAVE/WAVE.html
+- https://adm.ebu.io/background/what_is_the_adm.html
+- https://tech.ebu.ch/docs/tech/tech3285s7.pdf
+- https://tech.ebu.ch/docs/tech/tech3285s5.pdf
+
 ## xml
 Extensible Markup Language.
 

format/all/all.fqtest

@@ -85,6 +85,7 @@ cbor                 Concise Binary Object Representation
 csv                  Comma separated values
 dns                  DNS packet
 dns_tcp              DNS packet (TCP)
+dolby_metadata       Dolby Metadata (Atmos, AC3, Dolby Digital)
 elf                  Executable and Linkable Format
 ether8023_frame      Ethernet 802.3 frame
 exif                 Exchangeable Image File Format

format/format.go

@@ -92,6 +92,7 @@ var (
 	CSV                 = &decode.Group{Name: "csv"}
 	DNS                 = &decode.Group{Name: "dns"}
 	DNS_TCP             = &decode.Group{Name: "dns_tcp"}
+	Dolby_Metadata      = &decode.Group{Name: "dolby_metadata"}
 	ELF                 = &decode.Group{Name: "elf"}
 	Ether_8023_Frame    = &decode.Group{Name: "ether8023_frame"}
 	Exif                = &decode.Group{Name: "exif"}

format/riff/common.go

@@ -36,9 +36,9 @@ func riffDecode(d *decode.D, path path, headFn func(d *decode.D, path path) (str
 		}
 	})
 
-	wordAlgin := d.AlignBits(16)
-	if wordAlgin != 0 {
-		d.FieldRawLen("align", int64(wordAlgin))
+	wordAlign := d.AlignBits(16)
+	if wordAlign != 0 {
+		d.FieldRawLen("align", int64(wordAlign))
 	}
 }
 
@@ -58,6 +58,16 @@ var chunkIDDescriptions = scalar.StrMapDescription{
 
 	"dmlh": "Extended AVI header",
 
+	"data": "Raw sound encoded data",
+	"bext": "Broadcast extension, e.g. creator, date, etc.",
+	"smpl": "Sample metadata, e.g. loop points",
+	"fact": "Original info used for compression, e.g. sample length",
+
+	// BWF ADM master and Dolby Metadata
+	"chna": "Track UIDs of Audio Definition Model",
+	"axml": "Audio Definition Model ambisonics and elements",
+	"dbmd": "Dolby Metadata, e.g. Atmos, AC3, Dolby Digital [Plus]",
+
 	"ISMP": "SMPTE timecode",
 	"IDIT": "Time and date digitizing commenced",
 	"IARL": "Archival Location. Indicates where the subject of the file is archived.",