SOLVED How to copy extended attributes created by MacOS clients to other files?

tobiasbp · Feb 25, 2019

Hello there....

I'd like to change tags on files as created by my MacOS users.

I a share, I have two files tagged "red" and "green". This is what I get when listing the extended attributes:

Code:
lsextattr user * tagged_green DOSATTRIB org.netatalk.Metadata DosStream.com.apple.metadata[STRANGE_UNICODE_CHAR_HERE]kMDItemUserTags:$DATA tagged_red DosStream.com.apple.metadata[STRANGE_UNICODE_CHAR_HERE]kMDItemUserTags:$DATA DOSATTRIB org.netatalk.Metadata

[STRANGE_UNICODE_CHAR_HERE] is a place holder for a "weird" unicode character. The actual data?

Is there any way to read/set the tags/metadata in a usable way? Any suggestions much appreciated.

Thanks,
Tobias

tobiasbp · Mar 4, 2019

I now have a single directory created by a MacOS client and called "FROM_MAC_OS". The dir is tagged "green".

Let's see the extended attributes on the dir:

Code:

lsextattr user FROM_MAC_OS

FROM_MAC_OS    org.netatalk.Metadata    DosStream.com.apple.metadata_kMDItemUserTags:$DATA

OK. I think this means, that the dir has two extended attributes:

org.netatalk.Metadata
DosStream.com.apple.metadata_kMDItemUserTags:$DATA

I can get the data in org.netatalk.Metadata like this:

Code:

getextattr -s user org.netatalk.Metadata FROM_MAC_OS

FROM_MAC_OS    "\000\^E\^V\000\^B\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\00\000\000\000\^D\000\000\000\M^Z\000\000\000\000\000\000\00\000\000\^Ab\000\000\000\^P\000\000\000\011\000\000\000z\000\000\000\040\000\000\000\^N\000\000\^Ar\000\000\000\^D\M^@DEV\000\000\^Av\000\000\000\000\M^@INO\000\000\^A~\000\000\000\000\M^@SYN\000\000\^A\M^F\000\000\000\000\M^@SV~\000\000\^A\M^N\000\000\000\000\000\000\000\000\000\000\000\000\000\^D\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000$\^O\M-7d$\^O\M-7d\M^@\000\000\000$\^O\M-7d\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"

It's my impression, that the data I'm after, is in

DosStream.com.apple.metadata_kMDItemUserTags:$DATA

, so I'll read the data (Remember to put name in single quotes or escape the $):

Code:

getextattr -s user 'DosStream.com.apple.metadata\042_kMDItemUserTags:$DATA' FROM_MAC_OS
FROM_MAC_OS    "bplist00\M-!\^AWGreen\012\012\000\000\000\000\000\000\^A\^A\000\000\000\000\000\000\000\^B\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\^R"

Let's see, what happens if I change the tag from "green" to "red":

Code:

getextattr -s user 'DosStream.com.apple.metadata\042_kMDItemUserTags:$DATA' FROM_MAC_OS

FROM_MAC_OS    "bplist00\M-!\^AURed\012\012\000\000\000\000\000\000\^A\^A\000\000\000\000\000\000\000\^B\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\^P"

Great, I have the right extended attribute for sure, and I can read the data. Now, if I copied the data in the extended attribute in another dir, I should be able to change the tag of the other dir.

Get the original data (Like I did before):

Code:

getextattr -s user 'DosStream.com.apple.metadata\042_kMDItemUserTags:$DATA' FROM_MAC_OS

FROM_MAC_OS    "bplist00\M-!\^AURed\012\012\000\000\000\000\000\000\^A\^A\000\000\000\000\000\000\000\^B\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\^P"

Set the same data on currently untagged dir "UNTAGGED" (Remember single quotes on attribute name and data?):

Code:

setextattr user 'DosStream.com.apple.metadata\042_kMDItemUserTags:$DATA' 'bplist00\M-!\^AURed\012\012\000\000\000\000\000\000\^A\^A\000\000\000\000\000\000\000\^B\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\^P' UNTAGGED

Read the extended attribute we just set on dir "UNTAGGED":

Code:

getextattr -s user 'DosStream.com.apple.metadata\042_kMDItemUserTags:$DATA' UNTAGGED

UNTAGGED    "bplist00\134M-!\134^AURed\134012\134012\134000\134000\134000\134000\134000\134000\134^A\134^A\134000\134000\134000\134000\134000\134000\134000\134^B\134000\134000\134000\134000\134000\134000\134000\134000\134000\134000\134000\134000\134000\134000\134000\134^P"

It did not work, what I read from "UNTAGGED" is not what I have in "FROM_MAC_OS". Specifically, all occurrences of '\' have been translated to '\134'.

Let's see the difference between the original data and the copy in hex:

Code:

getextattr -x user 'DosStream.com.apple.metadata\042_kMDItemUserTags:$DATA' FROM_MAC_OS
FROM_MAC_OS    62 70 6c 69 73 74 30 30 a1 01 55 52 65 64 0a 36 08 0a 00 00 00 00 00 00 01 01 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 10

getextattr -x user 'DosStream.com.apple.metadata\042_kMDItemUserTags:$DATA' UNTAGGED
UNTAGGED    62 70 6c 69 73 74 30 30 5c 4d 2d 21 5c 5e 41 55 52 65 64 5c 30 31 32 5c 30 31 32 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 5e 41 5c 5e 41 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 5e 42 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 30 30 30 5c 5e 50

Any suggestion on how I can feed the data to setextattr in the right way? It seems to want a string. I can read the data as HEX, but setextattr needs a string. Could it be a ASCII/Unicode problem? Maybe I'm feeding ASCII when Unicode is expected?

Thanks,
Tobias

anodos · Mar 4, 2019

It's easier to interact with the DosStream.com.apple.metadata_kMDItemUserTags:$DATA streams from the command line if you add the "catia" module and set fruit:encoding = native.

The xattr itself is IIRC a plist of sorts, but probably the best way of manipulating it is to getextattr -qq user DosStream.com.apple.metadata:_kMDItemUserTags:$DATA | b64encode - to return a base64 encoded string. The -qq is required to remove the path from the output. Create separate strings for each color combination that you're concerned about, then experiment with piping the string through b64decode into setextattr -i (stdin).

tobiasbp · Mar 4, 2019

anodos said:
... but probably the best way of manipulating it is to getextattr -qq user DosStream.com.apple.metadata:_kMDItemUserTags:$DATA | b64encode - to return a base64 encoded string.

Great suggestion(s), I'll try it out straight away and report back. Thanks!

anodos · Mar 4, 2019

I've done this procedure many times to debug xattr behavior. If you run into problems, hexdump is your friend.

tobiasbp · Mar 4, 2019

tobiasbp said:
Great suggestion(s), I'll try it out straight away and report back. Thanks!

It works!

Extract the data in extattr org.netatalk.Metadata in file "TAGGED_GREEN" for later use:
getextattr -qq user org.netatalk.Metadata TAGGED_GREEN | b64encode - > org.netatalk.Metadata.green

Let's see the data (Because we can):
cat org.netatalk.Metadata.green

Code:

begin-base64 644 -
AAUWBwACAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAEAAAAmgAAAAAAAAAIAAABYgAAABAAAAAJAAAA
egAAACAAAAAOAAABcgAAAASAREVWAAABdgAAAACASU5PAAABfgAAAACAU1lOAAABhgAAAACAU1Z+
AAABjgAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAJA/b+CQP2/iAAAAAJA/b+AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAA
====

Let's make a dir (from the command line) to tag:
mkdir TAGGED_FROM_CLI

Add the "green" meta data to the dir "TAGGED_FROM_CLI":
b64decode -p org.netatalk.Metadata.green | setextattr -i user org.netatalk.Metadata TAGGED_FROM_CLI

MacOS client sees nothing new. Other MacOS user adds files to the dir holding "TAGGED_FROM_CLI". View updates. "TAGGED_FROM_CLI" is indeed tagged with color "green".

It seems like I don't need to copy the data in extattr DosStream.com.apple.metadata:_kMDItemUserTags:$DATA.

Thanks a lot @anodos

tobiasbp · Mar 4, 2019

Now, if only I could make my own plists (That's what I think is stored in the extattr's) with the combinations af tags/names i really want.....

anodos · Mar 4, 2019

tobiasbp said:
Now, if only I could make my own plists (That's what I think is stored in the extattr's) with the combinations af tags/names i really want.....

That's where you need to just hexdump various color tag xattrs to figure out the exact details of the on-disk format.

tobiasbp · Mar 4, 2019

anodos said:
That's where you need to just hexdump various color tag xattrs to figure out the exact details of the on-disk format.


diff hexdump.green hexdump.red

13c13
< 00000c0 4141 4141 4145 4141 4141 4141 4141 4141
---
> 00000c0 4141 4141 414d 4141 4141 4141 4141 4141
26,27c26,27
< 00001f0 4a41 2f41 2b62 5143 3250 692f 4141 4141
< 0000200 4a41 2f41 2b62 4141 4141 4141 4141 4141
---
> 00001f0 4a41 2f41 444f 5169 7a50 3667 4141 4141
> 0000200 4a41 2f41 444f 4167 4141 4141 4141 4141

It's probably the actual color, and the name... I'll investigate (some time). Thank you for all your help.

anodos · Mar 4, 2019

tobiasbp said:
diff hexdump.green hexdump.red 13c13 < 00000c0 4141 4141 4145 4141 4141 4141 4141 4141 --- > 00000c0 4141 4141 414d 4141 4141 4141 4141 4141 26,27c26,27 < 00001f0 4a41 2f41 2b62 5143 3250 692f 4141 4141 < 0000200 4a41 2f41 2b62 4141 4141 4141 4141 4141 --- > 00001f0 4a41 2f41 444f 5169 7a50 3667 4141 4141 > 0000200 4a41 2f41 444f 4167 4141 4141 4141 4141

It's probably the actual color, and the name... I'll investigate (some time). Thank you for all your help.

You can pipe into hexdump -C to see the canonical hex+ascii output.

tobiasbp · Mar 4, 2019

anodos said:
You can pipe into hexdump -C to see the canonical hex+ascii output.

Cool... Tell me more.. 5 colors to compare here. Looks to me like the unique tag data is in the two bytes here 00000090. How should I go about comparing the data (I've never actually used these tools before. So it' all new and exiting to me).


b64decode -p .org.netatalk.Metadata.yellow | hexdump -C
00000000  00 05 16 07 00 02 00 00  00 00 00 00 00 00 00 00  |................|
00000010  00 00 00 00 00 00 00 00  00 08 00 00 00 04 00 00  |................|
00000020  00 9a 00 00 00 00 00 00  00 08 00 00 01 62 00 00  |.............b..|
00000030  00 10 00 00 00 09 00 00  00 7a 00 00 00 20 00 00  |.........z... ..|
00000040  00 0e 00 00 01 72 00 00  00 04 80 44 45 56 00 00  |.....r.....DEV..|
00000050  01 76 00 00 00 00 80 49  4e 4f 00 00 01 7e 00 00  |.v.....INO...~..|
00000060  00 00 80 53 59 4e 00 00  01 86 00 00 00 00 80 53  |...SYN.........S|
00000070  56 7e 00 00 01 8e 00 00  00 00 00 00 00 00 00 00  |V~..............|
00000080  00 00 00 0a 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  00 00 24 0f e1 fc 24 0f  00 00 00 00 00 00 00 00  |..$...$.........|
000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000160  00 00 24 0f e4 14 24 0f  e4 14 80 00 00 00 24 0f  |..$...$.......$.|
00000170  e4 14 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000190

b64decode -p .org.netatalk.Metadata.red | hexdump -C
00000000  00 05 16 07 00 02 00 00  00 00 00 00 00 00 00 00  |................|
00000010  00 00 00 00 00 00 00 00  00 08 00 00 00 04 00 00  |................|
00000020  00 9a 00 00 00 00 00 00  00 08 00 00 01 62 00 00  |.............b..|
00000030  00 10 00 00 00 09 00 00  00 7a 00 00 00 20 00 00  |.........z... ..|
00000040  00 0e 00 00 01 72 00 00  00 04 80 44 45 56 00 00  |.....r.....DEV..|
00000050  01 76 00 00 00 00 80 49  4e 4f 00 00 01 7e 00 00  |.v.....INO...~..|
00000060  00 00 80 53 59 4e 00 00  01 86 00 00 00 00 80 53  |...SYN.........S|
00000070  56 7e 00 00 01 8e 00 00  00 00 00 00 00 00 00 00  |V~..............|
00000080  00 00 00 0c 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000160  00 00 24 0f ce 0e 24 0f  ce 0e 80 00 00 00 24 0f  |..$...$.......$.|
00000170  ce 0e 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000190

b64decode -p .org.netatalk.Metadata.green | hexdump -C
00000000  00 05 16 07 00 02 00 00  00 00 00 00 00 00 00 00  |................|
00000010  00 00 00 00 00 00 00 00  00 08 00 00 00 04 00 00  |................|
00000020  00 9a 00 00 00 00 00 00  00 08 00 00 01 62 00 00  |.............b..|
00000030  00 10 00 00 00 09 00 00  00 7a 00 00 00 20 00 00  |.........z... ..|
00000040  00 0e 00 00 01 72 00 00  00 04 80 44 45 56 00 00  |.....r.....DEV..|
00000050  01 76 00 00 00 00 80 49  4e 4f 00 00 01 7e 00 00  |.v.....INO...~..|
00000060  00 00 80 53 59 4e 00 00  01 86 00 00 00 00 80 53  |...SYN.........S|
00000070  56 7e 00 00 01 8e 00 00  00 00 00 00 00 00 00 00  |V~..............|
00000080  00 00 00 04 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000160  00 00 24 0f db f8 24 0f  db f8 80 00 00 00 24 0f  |..$...$.......$.|
00000170  db f8 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000190

b64decode -p .org.netatalk.Metadata.purple | hexdump -C
00000000  00 05 16 07 00 02 00 00  00 00 00 00 00 00 00 00  |................|
00000010  00 00 00 00 00 00 00 00  00 08 00 00 00 04 00 00  |................|
00000020  00 9a 00 00 00 00 00 00  00 08 00 00 01 62 00 00  |.............b..|
00000030  00 10 00 00 00 09 00 00  00 7a 00 00 00 20 00 00  |.........z... ..|
00000040  00 0e 00 00 01 72 00 00  00 04 80 44 45 56 00 00  |.....r.....DEV..|
00000050  01 76 00 00 00 00 80 49  4e 4f 00 00 01 7e 00 00  |.v.....INO...~..|
00000060  00 00 80 53 59 4e 00 00  01 86 00 00 00 00 80 53  |...SYN.........S|
00000070  56 7e 00 00 01 8e 00 00  00 00 00 00 00 00 00 00  |V~..............|
00000080  00 00 00 06 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  00 00 24 0f ea 51 24 0f  00 00 00 00 00 00 00 00  |..$..Q$.........|
000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000160  00 00 24 0f ea 7e 24 0f  ea 7e 80 00 00 00 24 0f  |..$..~$..~....$.|
00000170  ea 7e 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |.~..............|
00000180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000190

b64decode -p .org.netatalk.Metadata.gray | hexdump -C
00000000  00 05 16 07 00 02 00 00  00 00 00 00 00 00 00 00  |................|
00000010  00 00 00 00 00 00 00 00  00 08 00 00 00 04 00 00  |................|
00000020  00 9a 00 00 00 00 00 00  00 08 00 00 01 62 00 00  |.............b..|
00000030  00 10 00 00 00 09 00 00  00 7a 00 00 00 20 00 00  |.........z... ..|
00000040  00 0e 00 00 01 72 00 00  00 04 80 44 45 56 00 00  |.....r.....DEV..|
00000050  01 76 00 00 00 00 80 49  4e 4f 00 00 01 7e 00 00  |.v.....INO...~..|
00000060  00 00 80 53 59 4e 00 00  01 86 00 00 00 00 80 53  |...SYN.........S|
00000070  56 7e 00 00 01 8e 00 00  00 00 00 00 00 00 00 00  |V~..............|
00000080  00 00 00 02 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  00 00 24 0f ea 51 24 0f  00 00 00 00 00 00 00 00  |..$..Q$.........|
000000a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000160  00 00 24 0f ea 7e 24 0f  ea 7e 80 00 00 00 24 0f  |..$..~$..~....$.|
00000170  ea 7e 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |.~..............|
00000180  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*

There is also the other extattr DosStream.com.apple.metadata:_kMDItemUserTags:$DATA which I have not tried setting yet. I probably should, as I can se the data changing in it when I manipulate tags with MacOS clients:


getextattr -qq user 'DosStream.com.apple.metadata:_kMDItemUserTags:$DATA' TAGGED_FROM_CLI | hexdump -C
00000000  62 70 6c 69 73 74 30 30  a3 01 02 03 55 52 65 64  |bplist00....URed|
00000010  0a 36 58 4f 72 61 6e 67  65 0a 37 57 47 72 65 65  |.6XOrange.7WGree|
00000020  6e 0a 32 08 0c 12 1b 00  00 00 00 00 00 01 01 00  |n.2.............|
00000030  00 00 00 00 00 00 04 00  00 00 00 00 00 00 00 00  |................|
00000040  00 00 00 00 00 00 23                              |......#|
00000047

getextattr -qq user 'DosStream.com.apple.metadata:_kMDItemUserTags:$DATA' TAGGED_FROM_CLI | hexdump -C
00000000  62 70 6c 69 73 74 30 30  a1 01 57 47 72 65 65 6e  |bplist00..WGreen|
00000010  0a 32 08 0a 00 00 00 00  00 00 01 01 00 00 00 00  |.2..............|
00000020  00 00 00 02 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000030  00 00 00 12                                       |....|

anodos · Mar 4, 2019

This one is what you should be looking at: DosStream.com.apple.metadata:_kMDItemUserTags:$DATA

tobiasbp · Mar 4, 2019

anodos said:
This one is what you should be looking at: DosStream.com.apple.metadata:_kMDItemUserTags:$DATA

Should I not care about the other one?

Here is some info on how the data is formatted in _kMDItemUserTags.

tobiasbp · Mar 4, 2019

anodos said:
This one is what you should be looking at: DosStream.com.apple.metadata:_kMDItemUserTags:$DATA

Finder on MacOS clients do do not show the tags set on FreeNAS, unless I update extattr org.netatalk.Metadata.

anodos · Mar 4, 2019

tobiasbp said:
Finder on MacOS clients do do not show the tags set on FreeNAS, unless I update extattr org.netatalk.Metadata.

Yeah, this might be an easier process to perform through automation on a MacOS client.

tobiasbp · Mar 4, 2019

anodos said:
Yeah, this might be an easier process to perform through automation on a MacOS client.

I want to do on FreeNAS (And it works). But, if I don't set org.netatalk.Metadata it does not work.

Important Announcement for the TrueNAS Community.

SOLVED How to copy extended attributes created by MacOS clients to other files?

tobiasbp

Patron

tobiasbp

Patron

anodos

Sambassador

tobiasbp

Patron

anodos

Sambassador

tobiasbp

Patron

tobiasbp

Patron

anodos

Sambassador

tobiasbp

Patron

anodos

Sambassador

tobiasbp

Patron

anodos

Sambassador

tobiasbp

Patron

tobiasbp

Patron

anodos

Sambassador

tobiasbp

Patron

Similar threads