Error flashing the device on Linux

[mmaker]

hi,

I have a Chamelon RevE rebooted from lab401.
Running the flash command as provided in the wiki I get:

$ ./src/dfu-programmer atxmega32a4u flash --eeprom ~/dev/ChameleonMini/RevE/Firmware/Chameleon-Mini/Chameleon-Mini.eep 
Checking memory from 0x0 to 0x3F...  Empty.
0%                            100%  Programming 0x40 bytes...
[Device is write protected.
 X  ERROR
Memory write error, use debug for more info.

Following https://store.ryscc.com/blogs/news/upgrading-the-firmware-on-a-chameleonmini I then tried to use:

$ ./src/dfu-programmer atxmega32a4u erase
Checking memory from 0x0 to 0x7FFF...  Not blank at 0x7FF1.
Erasing flash...  Success
Checking memory from 0x0 to 0x7FFF...  Not blank at 0x7FF1.

Thinking that the flash would not be written if there was already non-zero data.

In both cases, the commands fail with an error. Where can I go from here?
I am using dfu-programmer revision 5254ee489f0048c36f1af660f2354a7746b3f8cc
Debian testing with libusb version 2:1.0.22-2, and lsusb gives me:

Bus 001 Device 067: ID 03eb:2fe4 Atmel Corp. ATxmega32A4U DFU bootloader

Meaning only the bootloader is left in my device :( and the WIndows GUI does not seem to recognize the Chamelon anymore. Is there any chance you could help me out with this?

[slurdge]

This is because the bootloader is different. From reverse engineering, it seems there is some protection. I would advice you to take a look at https://github.com/iceman1001/ChameleonMini-rebooted/blob/master/Software/Tools/crypt_operations.py and also try the --suppress-validation flag.
I think the operation you need would be either scramblebin or scramblehex.
You should be able to write a correct firmware this way. Beware that you are able to reboot in DFU by button press, because if you flash a bad firmware, you'll need this to get back on track.

Alternatively, you can try the Windows tools provided in this repository.

[mmaker]

Hi @slurdge, thanks a lot for your prompt response!
That's indeed very interesting (had a good laugh at the key and IV), however it's not very useful in my case:

$ sudo ./src/dfu-programmer atxmega32a4u flash-eeprom --debug 99999 --force --suppress-validation ~/dev/ChameleonMini-rebooted/Software/Tools/foo.bin 
     target: atxmega32a4u
    chip_id: 0x2fe4
  vendor_id: 0x03eb
    command: flash-eeprom
      quiet: false
      debug: 99999
device_type: XMEGA
------ command specific below ------
   validate: false
   hex file: /home/mu/dev/ChameleonMini-rebooted/Software/Tools/foo.bin

usb_set_debug: Setting debugging level to 99999 (on)
dfu.c:422: dfu_device_init( 1003, 12260, 0x7ffc727456a0, true, false )
usb_os_find_busses: Found 002
usb_os_find_busses: Found 001
usb_os_find_devices: Found 001 on 002
skipping descriptor 0x30
skipped 1 class/vendor specific endpoint descriptors
usb_os_find_devices: Found 072 on 001
usb_os_find_devices: Found 003 on 001
usb_os_find_devices: Found 002 on 001
skipping descriptor 0xFF
skipping descriptor 0xB
skipped 2 class/vendor specific endpoint descriptors
skipped 6 class/vendor specific interface descriptors
skipping descriptor 0x25
skipped 1 class/vendor specific endpoint descriptors
skipped 16 class/vendor specific interface descriptors
usb_os_find_devices: Found 001 on 001
error obtaining child information: Inappropriate ioctl for device
error obtaining child information: Inappropriate ioctl for device
error obtaining child information: Inappropriate ioctl for device
dfu.c:439: found device at USB:72,0
dfu.c:671: Found DFU Inteface: 0
dfu.c:301: dfu_abort( 0x7ffc727456a0 )
USB error: error sending control message: Broken pipe
dfu.c:892: dfu_abort ERR: -EPIPE: a) Babble detect or b) Endpoint stalled 0xffffffe0 (-32)
dfu.c:212: dfu_get_status( 0x7ffc727456a0, 0x7ffc72745644 )
dfu.c:238: ==============================
dfu.c:240: status->bStatus: errSTALLEDPKT (0x0f)
dfu.c:241: status->bwPollTimeout: 0x0000 ms
dfu.c:243: status->bState: dfuERROR (0x0a)
dfu.c:244: status->iString: 0x00
dfu.c:245: ------------------------------
dfu.c:696: State: dfuERROR (10)
dfu.c:260: dfu_clear_status( 0x7ffc727456a0 )
dfu.c:212: dfu_get_status( 0x7ffc727456a0, 0x7ffc72745644 )
dfu.c:238: ==============================
dfu.c:240: status->bStatus: OK (0x00)
dfu.c:241: status->bwPollTimeout: 0x0000 ms
dfu.c:243: status->bState: dfuIDLE (0x02)
dfu.c:244: status->iString: 0x00
dfu.c:245: ------------------------------
dfu.c:696: State: dfuIDLE (2)
Error reading line 1.
commands.c:405: Something went wrong with creating the memory image.

It seems to me that these errors happen writing the image, that they're not related to the input file I provide.
If I use "flash" istead of "flash-eeprom", using the hex file scrambled, I get exactly the same error.

On windows, the lab401 and your GUI do not recognize the usb device (Windows says it's "unknown" with driver "unknown").

Do you have any advice of where I should look at from here?

PS: your code is python3 only! Pull requests welcome? :^)

[slurdge]

@mmaker First, all the work here is done mainly by @iceman1001 and @bogiton if I'm not mistaken. I only happen to be interested by that bit.
The GUI will recognize the device only if it's operating normally.

Your DFU error is quite interesting, I've never seen any like that in my tests. The worst that happened was to me was flashing a non-working firmware. You should always use flash by the way, flash-eeprom is for .eep files.

Can you try to flash the original files with the BOOTLOADER_EXE way ?

As for python2, I'm quite sure a PR would be welcome. I just supposed everyone have some py3 laying around nowadays and I'm lazy :-p

[mmaker]

HI,

I am not sure what are the "original files" and what the BOOTLOADER_EXE way is. Could please be more precise?

Anyways, I'd like to add that if I try to flash using the dfu-programmer shipped with debin testing (0.6.1-1), I get the following error:

dfu.c:170: Message: m[1062] = 0x20
dfu.c:170: Message: m[1063] = 0x60
dfu.c:170: Message: m[1064] = 0x20
dfu.c:170: Message: m[1065] = 0x00
dfu.c:170: Message: m[1066] = 0x04
dfu.c:170: Message: m[1067] = 0x04
dfu.c:170: Message: m[1068] = 0x00
dfu.c:170: Message: m[1069] = 0x03
dfu.c:170: Message: m[1070] = 0x4d
dfu.c:170: Message: m[1071] = 0x46
dfu.c:170: Message: m[1072] = 0x5f
dfu.c:170: Message: m[1073] = 0x50
dfu.c:170: Message: m[1074] = 0x4c
dfu.c:170: Message: m[1075] = 0x55
dfu.c:170: Message: m[1076] = 0x53
dfu.c:170: Message: m[1077] = 0x31
dfu.c:170: Message: m[1078] = 0x4b
dfu.c:170: Message: m[1079] = 0x5f
dfu.c:170: Message: m[1080] = 0x37
dfu.c:170: Message: m[1081] = 0x42
dfu.c:170: Message: m[1082] = 0x00
dfu.c:170: Message: m[1083] = 0x00
dfu.c:170: Message: m[1084] = 0x00
dfu.c:170: Message: m[1085] = 0x00
dfu.c:170: Message: m[1086] = 0xd4
dfu.c:170: Message: m[1087] = 0x18
dfu.c:170: Message: m[1088] = 0x00
dfu.c:170: Message: m[1089] = 0x00
dfu.c:170: Message: m[1090] = 0x00
dfu.c:170: Message: m[1091] = 0x00
dfu.c:170: Message: m[1092] = 0x10
dfu.c:170: Message: m[1093] = 0x44
dfu.c:170: Message: m[1094] = 0x46
dfu.c:170: Message: m[1095] = 0x55
dfu.c:170: Message: m[1096] = 0x01
dfu.c:170: Message: m[1097] = 0x10
dfu.c:170: Message: m[1098] = 0xff
dfu.c:170: Message: m[1099] = 0xff
dfu.c:170: Message: m[1100] = 0xff
dfu.c:170: Message: m[1101] = 0xff
dfu.c:170: Message: m[1102] = 0xff
dfu.c:170: Message: m[1103] = 0xff
[ 0.017403] [00004a63] libusb: debug [libusb_alloc_transfer] transfer 0x55996d001470
[ 0.017407] [00004a63] libusb: debug [libusb_submit_transfer] transfer 0x55996d001470
[ 0.017410] [00004a63] libusb: debug [add_to_flying_list] arm timerfd for timeout in 20000ms (first in line)
[ 0.017421] [00004a63] libusb: debug [libusb_handle_events_timeout_completed] doing our own event handling
[ 0.017425] [00004a63] libusb: debug [handle_events] poll() 3 fds with timeout in 60000ms
[ 0.017833] [00004a63] libusb: debug [handle_events] poll() returned 1
[ 0.017844] [00004a63] libusb: debug [reap_for_handle] urb type=2 status=-32 transferred=64
[ 0.017847] [00004a63] libusb: debug [handle_control_completion] handling completion status -32
[ 0.017851] [00004a63] libusb: debug [handle_control_completion] unsupported control request
[ 0.017854] [00004a63] libusb: debug [disarm_timerfd] 
[ 0.017859] [00004a63] libusb: debug [usbi_handle_transfer_completion] transfer 0x55996d001470 has callback 0x7f9dd12792c0
[ 0.017862] [00004a63] libusb: debug [sync_transfer_cb] actual_length=64
[ 0.017866] [00004a63] libusb: debug [libusb_free_transfer] transfer 0x55996d001470
dfu.c:961: Unknown error 0xfffffff7 (-9)
atmel.c:1270: dfu_download failed. -9
atmel.c:1098: error flashing the block: -2
commands.c:367: Error while flashing. (-4)
Error while flashing.
[ 0.017901] [00004a63] libusb: debug [libusb_release_interface] interface 0
[ 0.017924] [00004a63] libusb: debug [libusb_close] 
[ 0.017929] [00004a63] libusb: debug [usbi_remove_pollfd] remove fd 9
[ 0.017936] [00004a63] libusb: debug [libusb_exit] 
[ 0.017939] [00004a63] libusb: debug [libusb_exit] destroying default context

[slurdge]

Look at this: https://github.com/iceman1001/ChameleonMini-rebooted/wiki/Flashing---windows and this https://github.com/iceman1001/ChameleonMini-rebooted/wiki/Flashing-Linux-(Unix)

[mmaker]

oh! Well well from what I recall when I tested on windows and launched that exe directly, I was getting:

dfu-old-driver: no device present.

after moving libusb0.dll to the same folder. :(

[securechicken]

Please try this : https://github.com/iceman1001/ChameleonMini-rebooted/wiki/Getting-started
An notably the last part... :)

[securechicken]

That's indeed very interesting (had a good laugh at the key and IV), however it's not very useful in my case:

BTW, I have a feeling this might be the "dxls" the key is talking about...
http://microchip.eefocus.com/module/forum/space-username-dxls.html

[slurdge]

Whoah, good find! Do you think we can contact this person ? I'm curious ...

[securechicken]

@slurdge Pure guess from a mere "dxls atmel" query on Gogole. Curious as well, but linked profile says last post is from end of 2016 (seems to match RevE works on emsec repo by the way) and nothing since. But from this repo page, "After talks with manufacturer, they also came to the conclusion that it should be open-sourced", so maybe @iceman1001 already chatted with him/them.

[iceman1001]

I sure did. We all know who did the hardware. He got dxls to make the gui and modify the firmware, and then afterwards Dxls went on with whatever s/he does. Among the questions I tried to get an answer to was about the encryption. Dxls had already forgotten the code and even the aes key. Back to square zero. This whole reverting of the sourcecode and enhancing it has been a long stretch of puzzling pieces together. The information gathered is all here on the issues / wiki, so it might be very confusing and unorganized. It is more a accurate picture of this project.

[0x023]

guys , any update or closure on this topic, unfortunately encountering the same issue as @mmaker with the same workflow described further down this topic.

[slurdge]

@0x023 , you should check that your device comes into dfu mode. This is done through the GUI or (if it works on your device), button combination. If you confirm that your device is in dfu mode (in device explorer) and even the .exe method doesn't work, then it means there's a new way of communicating. Your device comes from ?

[exander77]

@slurdge When I use scramblehex on firmware and eeprom, I get:

Error parsing the line.
Something went wrong with creating the memory image.

[slurdge]

@exander77 Can you provide me with the hex files? I'll look into it.

[exander77]

@slurdge Thank you: https://nextcloud.shy.cz/index.php/s/oaACo8fLB6XsN3H

I have set these variables:

TARGET=Chameleon-Mini
MCU=atxmega32a4u

I made it this way:

python3 ../Software/Tools/crypt_operations.py scramblehex $TARGET.eep $TARGET.eep.scramble
python3 ../Software/Tools/crypt_operations.py scramblehex $TARGET.hex $TARGET.hex.scramble

I am trying to flash it this way (/dev/ttyACM0 is my device):

echo -e "\rUPGRADE\r" > /dev/ttyACM0 && sleep 1
dfu-programmer $MCU get bootloader-version
dfu-programmer $MCU flash-eeprom --suppress-validation $TARGET.eep.scramble
dfu-programmer $MCU flash --suppress-validation $TARGET.hex.scramble
dfu-programmer $MCU reset

It goes to bootloader without problems and it gives me a version and it resets at the end. But both eeprom and firmware flashing gives me:

Error parsing the line.
Something went wrong with creating the memory image.

[slurdge]

Thanks. I'll look into it (however I'm on/off next week, sorry if I can't answer quickly).

[exander77]

@slurdge Not a problem, thank you very much!

[exander77]

@slurdge I have USBTinyISP, it may work as well. I will try it.

[slurdge]

@exander77 I looked at the files, and they seem correct. However I do not have my Chameleon with me so I can't try the full dfu-programmer.
Still, I noticed that I produced files that where LF terminated whereas the original hex files are CR terminated. Maybe try a unix2dos on the scramble file and tell me if it changes anything.

[slurdge]

@exander77 The error displayed is because the dfu-programmer is too old. Try with a recent version. It works for me with newer versions.

[securechicken]

Confirm this worked perfectly with the following sequence, on OSX and GNU/Linux, with dfu-programmer 0.7.2 (latest, either from Hombrew in OSX, or from Github repo in GNU/Linux), after putting the Chameleon in boot loader mode (black button pressed while plugging):

python3 ./Software/Tools/crypt_operations.py scramblehex ./Firmware/Chameleon-Mini/Chameleon-Mini.eep ./Firmware/Chameleon-Mini/Chameleon-Mini.eep.scramble
python3 ./Software/Tools/crypt_operations.py scramblehex ./Firmware/Chameleon-Mini/Chameleon-Mini.hex ./Firmware/Chameleon-Mini/Chameleon-Mini.hex.scramble
dfu-programmer atxmega32a4u erase --force
dfu-programmer atxmega32a4u flash --eeprom --force --suppress-validation ./Firmware/Chameleon-Mini/Chameleon-Mini.eep.scramble
dfu-programmer atxmega32a4u flash --force --suppress-validation ./Firmware/Chameleon-Mini/Chameleon-Mini.hex.scramble

It may be time to update wiki + Makefile and accept we have to stick the original bootloader and scrambling intermediary step for now; just to limit such issues.
By the was, this seems a duplicate of #11 .

[slurdge]

So happy it worked for you <3 !
Updating the wiki is on my todo list, just didn't take the time.

[securechicken]

Thanks a lot for this reverse engineering work. Too late, I made things clear in Linux flashing wiki page now, I think. I even guess these issues could be closed :)

[iceman1001]

We should do some shell scripts for easy flashing, like we have over at proxmark repos

[securechicken]

That is the purpose of PR #111: Makefile will endorse the "shell script" role, by just calling ones that exist already. "make dfu-prog" will build firmware and flash-it right away on OSX and GNU/Linux.

[iceman1001]

yes, but that PR removes backwards functionality...
a solution that supports both would be nice. And for most ppl, shell scripts seem to work better than dark woodo makefiles :)

[iceman1001]

@mmaker has your issues been resolved using the latest code as of today?

[iceman1001]

And, @ShinHub , email me.

[securechicken]

@iceman1001 oh I am going to be rumbled so much... I knew trying to get issues closed would get me in trouble :(

[securechicken]

@slurdge , sorry if I do not understand anything as this is quite new to me, but I may have a theory I'd like to have your opinion on.

• It seems like the original RevE Bootloader embeds a thing called "SPM driver", as we can see from "GenerateBootloader.bat" and "Makefile",
• this SPM driver is in fact built with application compilation ("sp_driver.h" and associated .s file, included from ChameleonMini.h - these are the exact sample codes provided by Atmel technote AVR1316 which provide a SPM library, except a section is changed from ".BOOT" to ".text"), although not necessary in app, and not called there,
• this looks like a hack that has been crafted because they did not manage to build that in bootloader code directly, maybe because of size constraint (Hence the change from ".BOOT", to ".text" in sp_helper.s compared to AVR1316? I am not sure to get them all, but I also think there are some arty defines of memory addresses at different places, like in Makefile and sp_driver.h, that may result in some sort of overlap between flash zones),
• as described in tech note AVR1316 on self-programming, the SPM can totally read and re-write all flash memory, including bootloader. Problem, SPM instructions can only be called from bootloader section. Solution, we can "jump" to bootloader from app (see "SystemInit" first part in System.c from application program), and to SPM function directly very probably, because we know where it is, and bootloader source code so how it will act when called,
• we may still have an issue with "Lock bits" and "SPM Lock" that would have been set somewhere to prevent any bootloader write (see AVR1316, 2.4 and notably 2.4.3), but we would probably be able to drop them if any.

So my theory is that one should be able to:

• compile a "proper" bootloader (we have source, or we have the compiled correctly PINed one),
• compile a special app that stuff this bootloader in Flash memory, in the Application data section for instance,
• from the special app, just like our ChameleonMini app, jump into bootloader, manage to trigger a SPM call to write (we know it's there, and theoretically where!) the bootloader that had been stuffed into app data section. You just have to get sure you do not overwrite the SPM part of bootloader by writing the new one from your app data section, or I am not sure of what could happen.
  Wouldn't that be feasible?

A part from that, I wonder if the strange "scramble" (not the AES round) that we have to reverse to flash an app from original RevE bootloader would not be related to the way the Flash is written (see AVR1316, title 2.2, atomic vs. split write. The second would need kind of XOR/ORing operations to write with SPM).

[slurdge]

@ShinHub No problems!
Your understanding is quite good, and the same as mine. I did come exactly with the same conclusions as you.

However, I think the problem is with the Lock bits. As per the manual (8331 4.16.6),
"Boot lock bit for the boot loader section.
Read and write lock – SPM is not allowed to write to the boot loader section,
and (E)LPM executing from the application section is not allowed to read
from the boot loader section.
If the interrupt vectors are placed in the application section, interrupts are
disabled while executing from the boot loader section."

I dumped the lock bits and they both are:

• prevent writes from programmer
• prevent writes from SPM

So; unless we find a workaround, it would be hard to unlock. IIRC some people did got around these limitations on some AtMels, but that is quite a bit of work. As for the XOR, I'm pretty sure it's not in the original bootloader and, if needed, wouldn't be done on the PC side.

[securechicken]

@slurdge thanks taking time answering my mashup. Great, I am happy it was not just my will, and that such a flow makes sense... I would have loved trying something like that. Thanks taking time checking with locks bits as well, as it was my next question. I'll continue to read here and there... I hope there is a way around this still, cause next step is looking for vulnerabilities to exploit in bootloader.

One last question though, how come the lock bits prevent a write from programmer, but you managed to write another bootloader with AVRISP, isn't it?

[securechicken]

@slurdge oh, now I get it. From the manual (8331, just where you pointed at) : "bits can only be written to a more strict locking. Resetting the bits is possible only by executing a chip erase command". And such a command, of course, is only (and always) possible with a hardware programmer like AVRISPmkII.

So can we say it is a dead end to program bootloader of ChameleonMini RevE rebooted as we get them with a mere software kungfu?

[slurdge]

@ShinHub Yes. BUT. Maybe the lock is incorrectly put on some devices. Maybe we can also find a way to sidestep the issue. However I didn't find right now any way to do that.

[securechicken]

@slurdge @iceman1001
Ok. I think that due to the lot of lessons learned here, recent changes to ease "locked boot loader" flashing from Makefile, and lack of track to follow on any other method for now, we should close this issue.
By the way, I confirm that compiling and AVRISPmkII-flashing the boot loader source code recently put in this repo and fixed with correct PINing, does unlock it for good and forever. Using make UNLOCKED_F=True dfu-prog works great.

[iceman1001]

@ShinHub is this documented on the wiki in a clear way?

[securechicken]

@iceman1001 Yes it is, plus the recent changes in Makefile to ease flashing in both locked and unlocked conditions.