Page 1 of 1

Fixed: HxD doesn't read entire partition? (misses final sectors?)

Posted: 03 May 2018 01:26
by hxdfan
version 1.7.7.0 - running on WINNT 4.0 (Windows NT 4.0)

HxD will not read the final sectors/clusters on a logical drive.

It seems okay with physical drives (e.g. HDD 0), but truncates logical drives (e.g. C:). I do not know if this is a WINNT 4.0 flaw.

So far as I can tell HxD truncates the logical drive by the same number of system-use sectors (i.e. Boot Sector, Fats #1 & #2, Root Dir).

For example, a 16gb FAT32 flash drive has 31258592 sectors. Of these, 32 sectors are hidden (MBR) so we're down to 31258560 sectors. Of these, 8192 are system sectors. And HxD only sees 31250368 sectors which is 8192 sectors short of the logical drive's true sector count.

Furthermore, the flash drive has 16,000,188,416 bytes - at least according to WINNT. Divide by 512, and we have 31250368 sectors which is the incorrect number of sectors HxD sees.

This problem happens regardless of file system (FAT16, FAT32) or disk size.

Is it possible that HxD is misreading the Boot Sector, and taking "Total Sectors" and subtracting "Hidden sectors" (correct) but also subtracting "Reserved sectors" and "Sectors per FAT" (incorrect)?

I also notice that HxD will not load a flash drive as a physical device, only a logical device. The disadvantage here is that we can't see the MBR sectors.

Sorry if I sound churlish. I appreciate anyone who takes the time to create a highly useful product.

Thank you in advance!

Re: HxD doesn't read entire partition? (misses final sectors?)

Posted: 10 May 2018 06:13
by Maël
Hi,

Have you tried HxD 2.0 RC (new section on http://mh-nexus.de) ?

Thanks

Re: HxD doesn't read entire partition? (misses final sectors?)

Posted: 21 Jun 2018 02:09
by hxdfan
Sorry for the delay.

Unfortunately I can not run the setup package on Winnt 4.0

I keep getting the message "HxDSetup.exe is not a valid Windows NT application."

Re: HxD doesn't read entire partition? (misses final sectors?)

Posted: 21 Jun 2018 13:11
by Maël
Oh yes, HxD 2.0 only works on Win XP and up (and HxD 1.7.7.0 wont be updated further).

What I can tell you is that HxD also considers the hidden sectors, when opening a partition as a physical drive (you have to set a special option for Windows to show all sectors, which HxD does).
But when it opens it as logical drive it will not show the hidden sectors and use the size reported by the file system. Otherwise there would be no real difference between a logical and a physical drive, if it hasn't more than one partition.

Usually that only affects the sectors *after* the partition, not the ones before it, like boot sectors.
That may be dependent on Windows version.

The one issue that remains is that there is no way to open a partition individually as a "physical" unit. Maybe it's time to revise that logic and naming. I actually started a prototype for a new disk open window, where you can see the whole organization of a disk and its partitions visually, a bit like in disk management of newer Windows NTs. If you are interested in testing this, let me know (though it will only be WinXP+, I am afraid).
I also notice that HxD will not load a flash drive as a physical device, only a logical device. The disadvantage here is that we can't see the MBR sectors.
This shouldn't happen and is somewhat confusing. You mean that same drive shows up only once, as logical drive, in the disk open window?
I remember that this also happens for floppy drives, which exist as physical and logical device on Win9x (well, actually I access the BIOS there). But WinNT does not expose a physical drive, because floppies are not supposed to have more than one partition.
Depending on how that flash drive "declares" itself to the system, it may not be listed as physical device and only appear as logical.

I checked the source code again, and the feature to include the hidden sectors was introduced in Windows XP, so I wont be able to change that.

The only thing that still is somewhat odd, is that you can't see the MBR, which is usually possible, for logical drives. This will never include all the partitioning information of the drive (that would be considered part of the physical drive), but the meta information that begins the specific partition/logical drive you open should be visible. Technically it is not the MBR but the volume boot record (VBR).

To clarify. Your title says it misses the final sectors (the hidden ones behind the partition), but does it also miss the boot sectors at the start of the partition? If it just missed the final sectors, but the physical drive (in HxD) shows them fine, then I might be able to make a patch to the exe file itself, to fix that. If the boot sector is truly missing at the beginning, then that's a limitation of WinNT and how it gives access to partitions.

Re: HxD doesn't read entire partition? (misses final sectors?)

Posted: 28 Jun 2018 01:41
by hxdfan
Thanks for replying. I think there's been a misunderstanding.

HxD doesn't miss the "final sectors (the hidden ones behind the partition)". It misses the final sectors at the end of the partition and occasionally the physical drive itself. I had wondered if this has to do with the end of a hard drive being a "partial last cylinder". I'll look into this tonight.

Re: HxD doesn't read entire partition? (misses final sectors?)

Posted: 19 Jul 2018 10:25
by Maël
Any news?

Re: HxD doesn't read entire partition? (misses final sectors?)

Posted: 01 Aug 2018 03:10
by Maël
To finally close this topic, I analyzed this problem with a 16 GB SD card.

The total physical sector count is 31.116.288 in this case (exact sizes depend on the model of the drive/card).

For HxD 2.0:
As a logical drive, the first 8192 are hidden compared to a physical drive. Besides these hidden leading sectors all the other sectors from the physical drive are also shown in the logical drive. This is as it should be for logical drives.

HxD 1.7.7.0 behaves the same, just that for the logical drive, additionally, sectors at the end are hidden.
This is because of what GetDiskFreeSpaceEx() returns, and is the normal way to determine a volume's size.
IOCTL_DISK_GET_LENGTH_INFO returns more accurate information, but is only available since Windows XP, so it wouldn't solve the issue for NT 4.0.

Maybe something like IOCTL_DISK_GET_PARTITION_INFO might have been used instead, but the current WinAPI documentation says it's only supported on XP+ as well (though the documentation was updated over time and the minimal Windows version were increased, even if the functions existed in earlier versions). Either way, HxD 2.0 supports XP+ only, so this will not change anymore for older Windows systems, and as far as I can see, there would not have been any way to determine the correct length of a volume on these systems.
Furthermore, the flash drive has 16,000,188,416 bytes - at least according to WINNT. Divide by 512, and we have 31250368 sectors which is the incorrect number of sectors HxD sees.
It is very likely the value GetDiskFreeSpaceEx() returns and what Windows Explorer uses as well, so that would make sense it matches with what HxD 1.7.7.0 shows for logical drives 16000188416/512 = 31250368. 31250368 +32+8192 = physical drive sector count.
So far as I can tell HxD truncates the logical drive by the same number of system-use sectors (i.e. Boot Sector, Fats #1 & #2, Root Dir).

For example, a 16gb FAT32 flash drive has 31258592 sectors. Of these, 32 sectors are hidden (MBR) so we're down to 31258560 sectors. Of these, 8192 are system sectors. And HxD only sees 31250368 sectors which is 8192 sectors short of the logical drive's true sector count.
Summary: I can confirm this issue for HxD 1.7.7.0, but as mentioned above it's unlikely I could have done it differently for WinNT 4.0.

HxD 2.0 does not have this issue, so I'll mark this as fixed.

(There is another issue limited to the disk open window, which shows a too big size for logical drives, but it does not affect the hex editor view. This will be fixed shortly, as well.)