Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

Strange daily security run error: ctl_datamove aborted

danb35

FreeNAS Wizard
Joined
Aug 16, 2011
Messages
10,324
Thanks
3,664
#1
I'm getting occasional messages in my daily security run email, and I'm having trouble deciphering them--either my Google-fu is weak, or there just isn't much out there on this. Here's the last one I got a couple of days ago:
Code:
ctl_datamove: tag 0x10000037 on (0:3:1) aborted
ctl_datamove: tag 0x10000075 on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 91 60 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000004c, type 1
(0:3:1/0): ctl_process_done: 107 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 b1 d0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000010, type 1
(0:3:1/0): ctl_process_done: 107 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 80 e3 48 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000069, type 1
(0:3:1/0): ctl_process_done: 108 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 7d a1 18 00 00 00 10 00 00 
(0:3:1/0): Tag: 0x10000079, type 1
(0:3:1/0): ctl_process_done: 110 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 81 63 78 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000061, type 1
(0:3:1/0): ctl_process_done: 133 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 81 6d f0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000036, type 1
(0:3:1/0): ctl_process_done: 133 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 81 6e 00 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000026, type 1
(0:3:1/0): ctl_process_done: 142 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 9a 2a 18 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000060, type 1
(0:3:1/0): ctl_process_done: 142 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 c0 50 e8 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000013, type 1
(0:3:1/0): ctl_process_done: 157 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 bd 27 90 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000050, type 1
(0:3:1/0): ctl_process_done: 157 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d4 d0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000004e, type 1
(0:3:1/0): ctl_datamove: 138 seconds
ctl_datamove: tag 0x1000004e on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 91 50 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000004a, type 1
(0:3:1/0): ctl_datamove: 138 seconds
ctl_datamove: tag 0x1000004a on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 25 14 12 40 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000020, type 1
(0:3:1/0): ctl_datamove: 138 seconds
ctl_datamove: tag 0x10000020 on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 32 28 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000006, type 1
(0:3:1/0): ctl_datamove: 138 seconds
ctl_datamove: tag 0x10000006 on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 25 13 18 18 00 00 00 18 00 00 
(0:3:1/0): Tag: 0x1000005a, type 1
(0:3:1/0): ctl_datamove: 138 seconds
ctl_datamove: tag 0x1000005a on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 94 e8 b8 00 00 00 10 00 00 
(0:3:1/0): Tag: 0x10000004, type 1
(0:3:1/0): ctl_datamove: 138 seconds
ctl_datamove: tag 0x10000004 on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d4 f0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000032, type 1
(0:3:1/0): ctl_datamove: 135 seconds
ctl_datamove: tag 0x10000032 on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d5 00 00 00 00 10 00 00 
(0:3:1/0): Tag: 0x10000067, type 1
(0:3:1/0): ctl_datamove: 135 seconds
ctl_datamove: tag 0x10000067 on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 91 60 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000014, type 1
(0:3:1/0): ctl_datamove: 138 seconds
ctl_datamove: tag 0x10000014 on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 b1 d0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000001a, type 1
(0:3:1/0): ctl_datamove: 138 seconds
ctl_datamove: tag 0x1000001a on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 80 e3 48 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000011, type 1
(0:3:1/0): ctl_datamove: 138 seconds
ctl_datamove: tag 0x10000011 on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d5 e0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000038, type 1
(0:3:1/0): ctl_datamove: 101 seconds
ctl_datamove: tag 0x10000038 on (0:3:1) aborted
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d3 68 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000058, type 1
(0:3:1/0): ctl_process_done: 200 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d4 d0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000004e, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 91 50 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000004a, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 25 14 12 40 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000020, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 32 28 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000006, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 25 13 18 18 00 00 00 18 00 00 
(0:3:1/0): Tag: 0x1000005a, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 94 e8 b8 00 00 00 10 00 00 
(0:3:1/0): Tag: 0x10000004, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d4 f0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000032, type 1
(0:3:1/0): ctl_process_done: 135 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d5 00 00 00 00 10 00 00 
(0:3:1/0): Tag: 0x10000067, type 1
(0:3:1/0): ctl_process_done: 135 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 91 60 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000014, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 b1 d0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000001a, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 80 e3 48 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000011, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d5 e0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000038, type 1
(0:3:1/0): ctl_process_done: 101 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 81 63 00 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000008, type 1
(0:3:1/0): ctl_process_done: 209 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d4 70 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000000e, type 1
(0:3:1/0): ctl_process_done: 156 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d4 88 00 00 00 10 00 00 
(0:3:1/0): Tag: 0x10000051, type 1
(0:3:1/0): ctl_process_done: 156 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d4 a0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000007d, type 1
(0:3:1/0): ctl_process_done: 156 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d3 b8 00 00 00 10 00 00 
(0:3:1/0): Tag: 0x10000015, type 1
(0:3:1/0): ctl_process_done: 182 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d3 e0 00 00 00 10 00 00 
(0:3:1/0): Tag: 0x1000006b, type 1
(0:3:1/0): ctl_process_done: 178 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 16 81 6c a8 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000017, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d3 48 00 00 00 10 00 00 
(0:3:1/0): Tag: 0x10000077, type 1
(0:3:1/0): ctl_process_done: 200 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d4 b8 00 00 00 10 00 00 
(0:3:1/0): Tag: 0x10000048, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 b2 c0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x10000022, type 1
(0:3:1/0): ctl_process_done: 211 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 3c e2 b2 c0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000002e, type 1
(0:3:1/0): ctl_process_done: 138 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d3 80 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000005e, type 1
(0:3:1/0): ctl_process_done: 183 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 46 cf d3 a0 00 00 00 08 00 00 
(0:3:1/0): Tag: 0x1000007e, type 1
(0:3:1/0): ctl_process_done: 182 seconds
(0:3:1/0): WRITE(16). CDB: 8a 00 00 00 00 00 25 12 87 60 00 00 00 10 00 00 
(0:3:1/0): Tag: 0x10000078, type 1
(0:3:1/0): ctl_process_done: 138 seconds

All the WRITE references would suggest to me that this is an issue with a disk operation, which would be cause for concern, but I'm not seeing any reference to a disk identifier. Thoughts?
 

dlavigne

Storage Engineering
Administrator
Moderator
iXsystems
Joined
May 24, 2011
Messages
11,460
Thanks
989
#3
What are you using for the initiator? Also, is the FreeNAS version the same as the one in your sig or 11.2-U4.1?
 

danb35

FreeNAS Wizard
Joined
Aug 16, 2011
Messages
10,324
Thanks
3,664
#4
The iSCSI initiator? That’s XCP-ng, a Citrix XenServer fork. And yes, I’m still on -U1.
 
Joined
Oct 15, 2015
Messages
3
Thanks
0
#5
Also having this issue as well. FreeNAS connected DAC to two ESXI servers running 6.7U2

Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daedd4 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daedd5 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daedd6 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daedd8 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daedd9 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daedda on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daeddb on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daeddd on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daeddf on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daede0 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daede1 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daede2 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daede4 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daede5 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daede6 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daede7 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daede9 on (1:3:0) aborted
Aug 24 15:29:58 SAN01 ctl_datamove: tag 0x7daedea on (1:3:0) aborted
 

danb35

FreeNAS Wizard
Joined
Aug 16, 2011
Messages
10,324
Thanks
3,664
#6
I'm continuing to see this occasionally on 11.2-U5 as well.
 
Joined
Oct 15, 2015
Messages
3
Thanks
0
#7
Some more logs if it helps
Code:
Aug 24 16:03:40 SAN01 (1:3:0/0): READ(10). CDB: 28 00 00 00 af 80 00 00 08 00
Aug 24 16:03:40 SAN01 (1:3:0/0): Tag: 0x7dc90bb, type 1
Aug 24 16:03:40 SAN01 (1:3:0/0): ctl_process_done: 138 seconds
Aug 24 16:03:40 SAN01 (1:3:0/0): READ(10). CDB: 28 00 00 00 af 80 00 00 08 00
Aug 24 16:03:40 SAN01 (1:3:0/0): Tag: 0x7dc90be, type 1
Aug 24 16:03:40 SAN01 (1:3:0/0): ctl_process_done: 98 seconds
Aug 24 16:03:40 SAN01 (4:3:1/1): WRITE SAME(16). CDB: 93 08 00 00 00 03 86 40 2d 00 00 00 04 00 00 00
Aug 24 16:03:40 SAN01 (4:3:1/1): Tag: 0x30000078, type 1
Aug 24 16:03:40 SAN01 (4:3:1/1): ctl_process_done: 193 seconds
Aug 24 16:03:40 SAN01 (4:3:1/1): WRITE(16). CDB: 8a 00 00 00 00 00 91 70 50 00 00 00 08 00 00 00
Aug 24 16:03:40 SAN01 (4:3:1/1): Tag: 0x30000073, type 1
Aug 24 16:03:40 SAN01 (4:3:1/1): ctl_process_done: 188 seconds
Aug 24 16:03:40 SAN01 (4:3:1/1): WRITE(16). CDB: 8a 00 00 00 00 00 91 70 48 00 00 00 08 00 00 00
Aug 24 16:03:40 SAN01 (4:3:1/1): Tag: 0x30000074, type 1
Aug 24 16:03:40 SAN01 (4:3:1/1): ctl_process_done: 188 seconds
Aug 24 16:03:40 SAN01 (4:3:1/1): WRITE(16). CDB: 8a 00 00 00 00 00 91 70 00 00 00 00 08 00 00 00
Aug 24 16:03:40 SAN01 (4:3:1/1): Tag: 0x30000014, type 1
Aug 24 16:03:40 SAN01 (4:3:1/1): ctl_process_done: 194 seconds
Aug 24 16:03:40 SAN01 (4:3:1/1): WRITE(16). CDB: 8a 00 00 00 00 00 91 70 10 00 00 00 08 00 00 00
Aug 24 16:03:40 SAN01 (4:3:1/1): Tag: 0x3000007f, type 1
Aug 24 16:03:40 SAN01 (4:3:1/1): ctl_process_done: 188 seconds
Aug 24 16:03:40 SAN01 (4:3:1/1): WRITE(16). CDB: 8a 00 00 00 00 00 91 70 08 00 00 00 08 00 00 00
Aug 24 16:03:40 SAN01 (4:3:1/1): Tag: 0x3000002b, type 1
Aug 24 16:03:40 SAN01 (4:3:1/1): ctl_process_done: 194 seconds
Aug 24 16:03:40 SAN01 (4:3:1/1): WRITE(16). CDB: 8a 00 00 00 00 00 91 6f f8 00 00 00 08 00 00 00
Aug 24 16:03:40 SAN01 (4:3:1/1): Tag: 0x30000002, type 1
Aug 24 16:03:40 SAN01 (4:3:1/1): ctl_process_done: 194 seconds
 
Joined
Oct 15, 2015
Messages
3
Thanks
0
#9
Great thanks!
 
Top