Cloning Bootable CFC's

[Dinosaur]

Hi All

I have a long history of cloning cfc's for machines that I made over the last
20 years or so.
WinImage worked a treat, untill I started using XPe and thus larger cfc's.

Now I use 2Gb Transcend 133x.
The system on the source disks are XPe (embedded) on one partition, and a second partition
for reading and writing data files. The system uses HORM to protect C: with all the XPe stuff
on it, and leaves D: unprotected so the machine can write updates to it.

Cloning these disks has been a real hit & miss affair.
At one stage just buying new cfc's to clone to solved the problem, but images that I have from those
disks wont always work even on new cfc's.

The cloning process will succeed on packages like HexEdit, RoadKill and HDD RawCopy.
The number of sectors copied is identical.
However when I boot the pc (Industrial cpu board) there are two likely scenarios.

1: It Boots and fires up XPe, but the D: is not visible.
When I use partition tools to look, I find that the partition is there, but is not "enabled" or has no drive name.
If I try to use HORM utility it reports that there are no "Fixed Disks" although this may be a bad translation
as the package is by Advantech from Taiwan.

2. It does not boot, and hangs after the statement "verifying DMI pool data ......"

I have gone around in circles trying different methods.
HORM disabled
HDDLFF low level format first.
CFC to File, then to CFC (.img)
CFC to CFC direct.
VHD to file then to CFC (.vhd)
Using XP on laptop instead of Win7 Pro.
Used RecoverRX which is a Transcend utility to format and recover bad CFC's.
Tried using FreeDos FDisk utility to correct partition, but as it has no name, cant select the partition.

Is there anyone on this forum that has succeeded in cloning CFC's as described above ?

Would really appreciate constructive suggestions on how to make this process foolproof.

Regards

[Dinosaur]

Hi All

Can't find an EDIT button for my previous post, so posted this new one.

It appears the problem is that the CFC is being treated like a removable disk.
Lots of Google rsults for making a USB CFC appear as fixed disk, but none work for me,
as my disks plug directly into the cpu board slot.

It also looks like the bit to be set for this is outside of the normal sector read/write area, as
the images that I have taken are working images, but when written to the same brand / size CFC
ends up as removable disk.

I have used HDDLLF and then the format utility from Transcend, but to no avail.

Does anyone have any idea as to how to change this bit on a TRanscend 133x 2Gb CFC ?
The other option is ofcourse, that the bit is set, but the cpu board wont accept it.
It is also the exact same cpu board as the one on which the images were read.

Regards

[Vulcan]

I don't use XPe on the embedded systems I work on (which isn't in DR), and your first posting would need more time to explain & clarify than I have available, so I'll just add a couple of comments.
IMHO you are describing multiple issues in your first posting, based on those multiple symptoms, but there's not enough detailed results for me to analyse further from the other side of the planet. Overall, if I was in your shoes, I would start by picking one symptom, with one CF card, and troubleshoot that. Mixing multiple issues / symptoms, until you are sure that they have just one cause (unlikely in your case IMHO), is a recipe for wasted time & effort, in my troubleshooting experience.

For example: Pick one CF card which "works" correctly. Clone the card into a file using any technique you like (although I thought there were specific MS tools for XPe work in this area). Zero-fill the CF card (i.e. fully erase it). Then reverse your cloning, from the file to the same CF card. Does that CF card still "work" (boot & run) correctly? If so, then stop wasting time looking at your cloning process. Then pick a different CF card and, if that behaves "wrongly" in some way, then investigate the detailed differences between those 2 cards (and just those 2 cards).

It appears the problem is that the CFC is being treated like a removable disk.

That's a common topic for discussion, when using CF cards with normal WinXP and XPe. As you are finding, not all CF cards are the same, and this is not the only possible difference.

It also looks like the bit to be set for this is outside of the normal sector read/write area

Correct. Specifically, this is part of the information reported by the CF card, in response to an ATA Identify Device command, and is reported by the CF card controller based on its private data. There is no standard way across all CF cards, for changing that (but see below).

I have used HDDLLF and then the format utility from Transcend, but to no avail.

Correct. Formatting a card doesn't affect the data reported by the card in response to an ATA Identify Device command.

Does anyone have any idea as to how to change this bit on a TRanscend 133x 2Gb CFC ?

That specific card - no. Transcend make several ranges of cards. Last time I checked, some of which report themselves are removable disks, some as fixed disks. SanDisk used to provide a utility to switch how their older CF cards reported themselves, but it was specific to their cards. SanDisk industrial cards (only available via OEM deals, not retail, from my experience some years ago) report themselves as fixed disk by default. You'll need to research this by looking at other people's comments on appropriate forums, or talking to the manufacturers. This webpage is just one example of the (many) discussions about this:

http://vampiric.us/index.php/articles/os/xpe/cfbootcomp

However I still doubt that this explains all of the behaviours you described initially.

There is also a WinXP "filter driver" (originally from Hitachi, IIRC) which can make any CF card which claims to be removable in its Identify Device response, actually be treated by WinXP as if it had reported that it is a fixed disk. I have no idea if that is valid for use on XPe.

Hope those comments help. It's up to you whether you follow those suggestions

[Dinosaur]

Hi All

Vulcan, many thanks for the time and trouble to reply to my questions.

Hope those comments help. It's up to you whether you follow those suggestions

Always take constructive suggestions on board.

A:
Using one of my good cfc's I did the following.
1.Using WinHex (not HexEdit as previously stated) read the disk into a .img file
2.Did HDDLLF of that cfc
3.Using WinHex wrote the previously read .img back to it.
4.Put into target machine and got no further then "verifying DMI pool data ......"

B:
Using another of my good cfc's
1.Using WinHex read into .img file
2.Using WinHex wrote it straight back again.
3.Put into Target machine and all OK.

C:
Using the same cfc as in B:
1.Using RecoverRx from Transcend, tried to fully format the cfc,
but got an error.
2.Using Win7 formatted the cfc to Fat
3.Using RecoverRx and fully formatted the cfc
4.Using WinHex wrote previous .img to cfc
5.Put into Target machine and all OK

D:
Then tried a number of the 14 cfc's that I have used HDDLLF on
1.Using WinHex wrote the same .img to it
2.Placed into target machine
3.XPe booted but partition not visible, and HORM advises "No Fixed Disk"
OR
1.Formatted with Win7 to Fat
2.Using RecoverRx and formatted cfc
3.Using WinHex wrote previous .img to it
4.XPe booted but partition not visible, and HORM advises "No Fixed Disk"

My conclusions from this are as follows.
1.HDDLLF is destroying the "Fixed Disk bit"
2.RecoverRx will not format the cfc's if it is formatted with NTFS
3.Once the cfc is "Removable" there seems no way to reset it without
special software from Transcend. I have emailed them in the usa but after
two days no response.

Interestingly,
1.WinHex is reading 3,915,072 sectors
2.HDDLLF is zeroing 3,915,072 sectors.
3.WinHex is re-writing 3,915,072 sectors

Yet there is no recovery after HDDLLF.

Any suggestions or observations welcomed.

Regards

[Vulcan]

Thanks for the update, although I just see more apparent contradictions...

My conclusions from this are as follows.
1.HDDLLF is destroying the "Fixed Disk bit"

As you said before (and as I expanded in my previous reply) that "bit" (i.e. part of the Identify Device response) is outside of the user data area, which is overwritten by the LLF tool. Therefore I cannot see how running the LLF tool can change it, based on my experience (but of course I don't have your CF cards to try).

As I said before, I'm concerned that you have multiple problems since you're describing multiple different symptoms. Therefore I'm just going to focus on your test A in more detail. You said that after running steps 1-3, step 4 was:

4.Put into target machine and got no further then "verifying DMI pool data ......":

... which you then linked to the "Fixed Disk bit" being "destroyed" as your later conclusion.

I don't have your motherboard, and nowadays I don't use XPe (both of which are off-topic for this forum anyway), so I have to ask: How do you know that a CF card which reports itself as removable (i.e. not fixed) causes that behaviour of your boot process not progressing (or at least not progressing correctly) after the message of "verifying DMI pool data"? Have you got some conclusive evidence from previous tests where you changed a previously working CF card from reporting itself as "fixed disk" to "removable disk", using a CF card manufacturer-supplied utility (and that is the only change you made), to know that this is how your system responds in that exact situation? Personally I expect something else is happening, unrelated to your fixed/removable concern.

Note that you actually presented data which seem to show that the LLF tool does not cause the behaviour of the boot stopping after the BIOS reporting "verifying DMI pool data", because you said in test D:

Then tried a number of the 14 cfc's that I have used HDDLLF on [...] XPe booted

So therefore you have already shown with that result, that running the LLF tool does not cause the symptom which you reported at step A4, which you later attributed to running LLF tool.

As I said before, I would take just one failing test, and focus on that since the different test results are causing confusion, so that you didn't spot the contradiction between the result A4 being blamed on the LLF tool, and your result D3 (later D4) which seems to contradict that conclusion. Many troubleshooting approaches are possible, depending on which single test result you choose to concentrate on first. I also expect that you need more advice from people who use XPe and HORM utility, which isn't this forum... Of course that's just my view, other people may have different views for you to consider.

Personally I would start by using hdparm on Linux to collect the full Identify Device data (including the fixed/removable attribute) from all your CF cards, and use that technique to confirm that running LLF tool does not change that attribute, so that you can at least remove that from your list of hypotheses.

[Dinosaur]

Hi All

Vulcan, once again I appreciate the time taken.
To complicate things, Transcend have replied to me, and advises:

CF133 is a commercial CF card thus is a removable disk under all modes.

If you want to install OS/create partition/use as a bootable disk then please use our industrial CF cards.

All of the Transcend's industrial CF cards may operate as a fixed disk.

Commercial CF cards: CF133, CF400, CF600

Industrial CF cards: CF80, CF150, CF160, CF100I, CF200I, CF300

So, being quite bold I replied:

> Dear Customer,
>
> First of all, many thanks for your letter.
>
> CF133 is a commercial CF card thus is a removable disk under all modes
WRONG
I have been buying this card for 5 years, and have ALWAYS been able to use them as Fixed Disk.
That is straight out of the box, write an image to it with WinHex or WinImage.
I have some 20 machines running in the field with this disk with XPe as C: and partition as D:
Advantech EWF Manager reports this disk as "Fixed Disk"
Only when I did a Low Level Format did it become Removable.

Now, reading your reply, I may be wrong about the LLF stuffing up the Fixed Disk bit.
But consider this:
Good CFC reports as Fixed Disk,
Boots and allows access to partition.
EWF Manager allows protection.

Now I do a HDDLLF on this same CFC
Format it with Win7 (Have tried doing this with XP machine)
Write the previously saved image to it.
Boots but doesn't allow access to partition
EWF Manager won't allow protection "No Fixed Disk"

This CFC is now no longer retrievable, doesnt matter what I do to it,
it will not allow access to partitions.That is what drew me to that conclusion.
I agree that there are other factors at play.

The issue with:

4.Put into target machine and got no further then "verifying DMI pool data ......"

can be explained by the fact that doing a HDDLLF and then writing an image without formatting
or copying MBR to it. The moment I Format the CFC, every disk will then proceed to boot.
But depending on whether I have used LLF, it will allow access to partitions or not.

I have one Sandisk Ultra II which showed the same symptoms, but using the ATCFWCHG.com
managed to reset it to Fixed Disk.

So, without dragging this out to become an epic, other users should be aware that:
1.Using HDDLLF without formatting can prevent Transcend 133x cfc from booting succesfully.
2.Transcend is doing the same as SanDisk and not supporting Commercial CFC's for Fixed Disk operation,
and perhaps even setting them to "Removable" now, where they didn't before.

Guess I now have 14 cfc's that can only be used as single partition, and stop bitching
about $210 and start buying Industrial ones.
Unless Transcend come back with a write utility, I would still like to know where the "Fixed Disk" bit is
so that perhaps I can use debug or something similar to edit that address.

Once again , many thanks for your advise and suggestions.

Regards

[Vulcan]

Thanks for the update. I don't understand some parts of what you've said, but that doesn't matter now. To respond to your final question:

Unless Transcend come back with a write utility, I would still like to know where the "Fixed Disk" bit is

As I've said before, this "bit" is reported by the CF firmware, in the response to the ATA Identify Device command (encoded within Word 0 of that response, IIRC). You will need to research some more about ATA commands (as used by CF cards in "True IDE mode"), if you want to understand about the Identify Device (sometimes called Identify Drive) command (opcode 0xEC) and its response data.

so that perhaps I can use debug or something similar to edit that address.

It's not that simple. You would need to write your own equivalent to SanDisk's utility which you mentioned, using proprietary knowledge of whatever mechanism Transcend include in their CF firmware for changing this private CF firmware data - if they include one! Access to these internal parameters is not standardised, and hence different manufacturers are at liberty to use their own proprietary (i.e. secret) techniques for changing them. Since you don't have that information from Transcend, you are looking at a serious reverse-engineering challenge, or else persuading someone in Transcend to give you that information etc.

I hope that helps you to understand why this option is not as simple as implied by the suggestion to "use debug".

Anyway, good luck!

[Vulcan]

Just as an FYI, I saw these pages which discuss some of the issues involved in finding CF cards that can be used for booting, highlighting some of the differences between different cards. I thought you might find them interesting in your investigation, if you didn't see them already:

http://www.versalogic.com/kb/KB.asp?KBID=1393
http://www.versalogic.com/kb/KB.asp?KBID=1403
http://www.versalogic.com/kb/KB.asp?KBID=1640

[einstein9]

I guess this MIGHT solve your issue here

Am using it btw

hope will work with u

good luck

Attachments

[Vulcan]

That's an interesting idea (depending on available physical space for the adapter etc. of course).

To help with however much of the OP's problem is being caused by the fixed vs. removable issue (and I remain unsure whether that is 100% of the OP's problem(s) or not), a colleague gave me the below link to an "active" CF-to-IDE adapter which is advertised, that claims to always report that the CF card is a "fixed disk" to any OS, by modifying the card's response to the Identify Device command, before passing it to the OS. However, as explained in the links I gave above, there are several other possible issues, and without specific test results from the OP's cards, this type of adapter might be a waste of time.

In case it's of interest to the OP, here's the link I was given - note that neither I, nor my colleague, have tried this adapter; he was just aware of it and gave me the link when I mentioned this topic:

http://www.jschoenfeld.com/products/trueide_e.htm

[Dinosaur]

Hi All

May thanks for the replies about IDE adapters.
Prior to using cpu boards with on-board adapters, I used these boards of which I still have
quite a few.

CFCadapt.jpg (63.77 KiB) Viewed 25685 times

The jumper on it allows it to be set as the primary HDD.
However, this does not solve the problem, which is:
It boots Xpe, but as removable, and won't allow access to partitions.

Quite a bit of reading on those links,Thank you.

Regards

[Vulcan]

The jumper on it allows it to be set as the primary HDD.
However, this does not solve the problem

Correct, it cannot solve the problem which you describe (whether that is the only problem or not is a different subject, which I'm not touching in this reply).

Your photo is an example of typical passive CF-to-IDE adapters, which do not (and cannot) change whether a CF card reports itself as removable or fixed disk (that jumper which you mentioned relates to an interface signal, and not to what you call the "fixed bit" in the Identify Device response). That's what is different with the active CF-to-IDE adapter which I linked.

[einstein9]

Dear Vulcan.

it does WORK btw, some time ago i was using it in VoIP machines, with 256MB card,
booting the machine in Linux with the VoIP app.

the image i sent is not the adapter i ment, but i was referring to the IDEA only

good luck

[Dinosaur]

Hi All

Einstein9, I believe that there are two differences.
1.The adapters you show have active components that can report the disk plugged in as Fixed.
2.Linux.

I have no experience with Linux, but have read that it will allow access to partitions
on removable disks, whereas Windows XPe won't.

Regards

[jeremyb]

I guess this MIGHT solve your issue here

Am using it btw

hope will work with u

good luck

Have SD->IDE readers ever worked for you for data recovery? I've never had any luck w/ them.