Feature Request: Add the ability to detect modified sectors like KryoFlux's DTC.exe

[Digitoxin1]

My current workflow when dumping requires the use of both GW.exe and KryoFlux's DTC.exe.

I still need to use DTC.exe since it will detect and let me know which tracks on a disk have modified sectors.

It would be great if Greaseweazle had this feature so I could remove DTC.exe from my workflow and not have to deal with it anymore.

[keirf]

That's a good idea. What sort of report do you get from DTC?

[Digitoxin1]

This is the output I get from DTC.exe. Notice on track 00.0, it shows a *H +3 and on track 26.0, that it shows a *H +1 at the end of the log line. The +3 and +1 signify the number of sectors in the track it has determined were modified. The KryoFlux documentation describes this exception as follows:

*H Header extra data was found. Data is hidden in unused parts of the block header. Sector images can’t hold such data; warning only

The only warning that normally affects dumping quality is *T. Another one is *H, which is usual for modified data or protection, but sometimes (very rarely) it happens when the data is very hard to read, so the bitcells get delayed. You will always see *H on XROM protected disks and always see *H on modified tracks.

It would be great if GW.exe went one step further and let us know exactly which sectors were detected as modified instead of just providing a count.

KryoFlux DiskTool Console, v3.00_Win64, uiv.1, Apr 15 2018, 23:44:54
(c) 2009-2018 KryoFlux Products & Services Ltd.
Developed by The Software Preservation Society, www.softpres.org
Licensed for private, non-commercial use only.

Stream file: 3-D Helicopter Simulator (1989) (v2.0BH) (360K) [M 0.0,26.0]\track

00.0    : MFM: OK*, trk: 000, sec: 9, *H +3
00.1    : MFM: OK, trk: 000, sec: 9
01.0    : MFM: OK, trk: 001, sec: 9
01.1    : MFM: OK, trk: 001, sec: 9
02.0    : MFM: OK, trk: 002, sec: 9
02.1    : MFM: OK, trk: 002, sec: 9
03.0    : MFM: OK, trk: 003, sec: 9
03.1    : MFM: OK, trk: 003, sec: 9
04.0    : MFM: OK, trk: 004, sec: 9
04.1    : MFM: OK, trk: 004, sec: 9
05.0    : MFM: OK, trk: 005, sec: 9
05.1    : MFM: OK, trk: 005, sec: 9
06.0    : MFM: OK, trk: 006, sec: 9
06.1    : MFM: OK, trk: 006, sec: 9
07.0    : MFM: OK, trk: 007, sec: 9
07.1    : MFM: OK, trk: 007, sec: 9
08.0    : MFM: OK, trk: 008, sec: 9
08.1    : MFM: OK, trk: 008, sec: 9
09.0    : MFM: OK, trk: 009, sec: 9
09.1    : MFM: OK, trk: 009, sec: 9
10.0    : MFM: OK, trk: 010, sec: 9
10.1    : MFM: OK, trk: 010, sec: 9
11.0    : MFM: OK, trk: 011, sec: 9
11.1    : MFM: OK, trk: 011, sec: 9
12.0    : MFM: OK, trk: 012, sec: 9
12.1    : MFM: OK, trk: 012, sec: 9
13.0    : MFM: OK, trk: 013, sec: 9
13.1    : MFM: OK, trk: 013, sec: 9
14.0    : MFM: OK, trk: 014, sec: 9
14.1    : MFM: OK, trk: 014, sec: 9
15.0    : MFM: OK, trk: 015, sec: 9
15.1    : MFM: OK, trk: 015, sec: 9
16.0    : MFM: OK, trk: 016, sec: 9
16.1    : MFM: OK, trk: 016, sec: 9
17.0    : MFM: OK, trk: 017, sec: 9
17.1    : MFM: OK, trk: 017, sec: 9
18.0    : MFM: OK, trk: 018, sec: 9
18.1    : MFM: OK, trk: 018, sec: 9
19.0    : MFM: OK, trk: 019, sec: 9
19.1    : MFM: OK, trk: 019, sec: 9
20.0    : MFM: OK, trk: 020, sec: 9
20.1    : MFM: OK, trk: 020, sec: 9
21.0    : MFM: OK, trk: 021, sec: 9
21.1    : MFM: OK, trk: 021, sec: 9
22.0    : MFM: OK, trk: 022, sec: 9
22.1    : MFM: OK, trk: 022, sec: 9
23.0    : MFM: OK, trk: 023, sec: 9
23.1    : MFM: OK, trk: 023, sec: 9
24.0    : MFM: OK, trk: 024, sec: 9
24.1    : MFM: OK, trk: 024, sec: 9
25.0    : MFM: OK, trk: 025, sec: 9
25.1    : MFM: OK, trk: 025, sec: 9
26.0    : MFM: OK*, trk: 026, sec: 9, *H +1
26.1    : MFM: OK, trk: 026, sec: 9
27.0    : MFM: OK, trk: 027, sec: 9
27.1    : MFM: OK, trk: 027, sec: 9
28.0    : MFM: OK, trk: 028, sec: 9
28.1    : MFM: OK, trk: 028, sec: 9
29.0    : MFM: OK, trk: 029, sec: 9
29.1    : MFM: OK, trk: 029, sec: 9
30.0    : MFM: OK, trk: 030, sec: 9
30.1    : MFM: OK, trk: 030, sec: 9
31.0    : MFM: OK, trk: 031, sec: 9
31.1    : MFM: OK, trk: 031, sec: 9
32.0    : MFM: OK, trk: 032, sec: 9
32.1    : MFM: OK, trk: 032, sec: 9
33.0    : MFM: OK, trk: 033, sec: 9
33.1    : MFM: OK, trk: 033, sec: 9
34.0    : MFM: OK, trk: 034, sec: 9
34.1    : MFM: OK, trk: 034, sec: 9
35.0    : MFM: OK, trk: 035, sec: 9
35.1    : MFM: OK, trk: 035, sec: 9
36.0    : MFM: OK, trk: 036, sec: 9
36.1    : MFM: OK, trk: 036, sec: 9
37.0    : MFM: OK, trk: 037, sec: 9
37.1    : MFM: OK, trk: 037, sec: 9
38.0    : MFM: OK, trk: 038, sec: 9
38.1    : MFM: OK, trk: 038, sec: 9
39.0    : MFM: OK, trk: 039, sec: 9
39.1    : MFM: OK, trk: 039, sec: 9
40.0    : MFM: <unformatted>
40.1    : MFM: <unformatted>
41.0    : MFM: <unformatted>
41.1    : MFM: <unformatted>

Enjoy your shiny new disk image!
Please consider helping us to preserve media and continue development:
www.softpres.org/donate

[Digitoxin1]

Since gw.exe already outputs a nice full map of all of the sectors, maybe put something like an M on the grid when a modified sector is detected.

[keirf]

Okay this is mostly an exercise in plumbing I think. Definitely doable!

[Breiztiger]

hi no news for this enhancement ?

[philpem]

I feel like chipping in with a "watch out" here - if you don't care for the gory details then skip to the end for the issue :)

Back when I was actively working on DiscFerret and the Merlin disc analyser/decoder, I was looking at how this could be done (I wondered how SPS did it but never looked at their code, only the public descriptions - it would have been silly to encumber the DiscFerret code).

I came up with two ways to identify modification - but they only work on commercially-duplicated (using a Copypro, Trace or similar single-pass copier):

• Different gap width between the sector header and sector data.
• Timing change in the sector data

The gap width (number of bits of gap) should be a reasonable indicator. It works because a single-pass disc duplicator doesn't do separate format and write steps. Instead, it builds an image of the track, writes it, then reads the data back and verifies it. This all happens in two rotations of the disc.

The issue is that this would only work with discs which were made on a single-pass duplicator. If someone had created the discs on their Acorn, Amiga or whatever - it'll give a false "modified" indication. At the time, I was dealing with Acorn discs and most of them were duplicated on physical machines.

The (partial) solution I came up with was to look for differences in motor speed between the drives. The way I did it was to figure out the nominal bit cell width, normalise each measurement to that (divide by 1.0, 1.5 or 2.0), then calculate the timing error vs. ideal in PPM. In theory all sectors written with the same drive (i.e. machine) should be written at the same speed.

The error terms (drive figures are from http://www.techtravels.org/wp-content/uploads/pefiles/SAMSUNG-SFD321B-070103.pdf) boil down to:

• +/- 50ppm - disc controller clock crystal
• +/- 1.5% LSV speed variation (= +/- 15,000 ppm)
• +/- 1.5% ISV speed variation (= +/- 15,000 ppm)

ISV is instantaneous speed variation. LSV is long-term speed variation. Note that the Sony MPF-920 quotes +/- 3% ISV.

The ISV should (in theory) average out over the sector, leaving the LSV - which is the error in calibration for that particular drive. In actuality what's measured is the ISV+LSV of the reading drive, and the ISV+LSV of the writing drive.

If desired, the LSV of the reading drive can be measured (index pulse timing) and subtracted from the measurement.

What's left over is the LSV(write) + ClockError(write) + LSV(read) + ClockError(greaseweazle).

The clock error is only 100ppm and fades into the noise of the potentially 30,000 PPM motor speed error.

Once you've got the sector and sector header timing (and gap timings if you want to do that), you can analyse the timings to find out:

• Written with a single pass duplicator and not modified later: All sectors, headers, and gaps are the same speed. Gaps are also consistent. Only one write splice, near the index pulse.
• Formatted in the factory then written on a computer: Formatted elements (gaps and headers) have one timing value. Some or all sectors have different timing values.
• Formatted and written on the same computer: Formatted elements (gaps and headers) and sectors have the same or similar timing. Evidence of write splices or gap lengths which don't match the format.
• Data modified on a different computer: At least one sector with different timing to others on the disc.

Identifying a factory formatted (or duplicated) disc might be tricky, but I'd expect the motor speed to be pretty much dead on. They would have been checked and calibrated - if the distribution house cared about quality. There's also a chance there might be duplication metadata (TRACEBACK or similar) on track 80.