SOLVED Replications stopped after installing new boot device

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
On May 21 I installed a SSD for the FreeNAS OS and loaded it on. It all went fairly smoothly, and all my configuration is intact and everything working smoothly. However, recently I noticed that the last snapshots to be replicated were on May 20 (although in Storage > Snapshots the date column sorting does not work, so it's tedious to go through).

I looked at the replication task settings and they seem to be correct still. Snapshots seem to be up to date on the source. How can I troubleshoot this?
 

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
It turns out that after reinstalling the OS on the new boot device, the replication keys were messed up so I had to redo all the key stuff. Not sure why that configuration didn't copy over with the config file.

More importantly, it's disturbing that replication could fail silently, with no notification. If I hadn't searched through the snapshots on the destination pool I'd have never known.
 

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,924
More importantly, it's disturbing that replication could fail silently, with no notification.
Have you got alerts set up and tested in Alerts Services and Alerts Settings?
 

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
Good question. Destination email is set up in root user. System > Email is set up and sending a test email works. System > Alert Settings has all alert notifications set to immediate.

The problem may be in System > Alert Services > E-mail. It is enabled and has the correct destination email address (same as set for root user), but sending a test alert fails. No idea why.
 

Redcoat

MVP
Joined
Feb 18, 2014
Messages
2,924
Hmmmmm - those are all the checks I would have made, too. Mine is set up the same way. Maybe a reboot if you've made changes and not done so since?
 

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
OK I'll try that. But right now the replications seem to be copying over the entire datasets. I guess that's because the old snapshots are all being deleted because they are older than two weeks, so it is like starting fresh. Still don't fully understand snapshots. But I'll wait till it's done and try a reboot.
 

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
I looked through about 10 different logs before I found this alert error in middlewared.log. Have no idea what it means.
Code:
[2019/06/10 21:02:48] (ERROR) AlertServiceService.test():633 - Error in alert service 'Mail'
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/alert.py", line 631, in test
    await alert_service.send([test_alert], [], [test_alert])
  File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/../alert/service/mail.py", line 23, in send
    "text": await self._format_alerts(alerts, gone_alerts, new_alerts),
  File "/usr/local/lib/python3.6/site-packages/middlewared/alert/base.py", line 135, in _format_alerts
    return format_alerts(product_name, hostname, node_map, alerts, gone_alerts, new_alerts)
  File "/usr/local/lib/python3.6/site-packages/middlewared/alert/base.py", line 183, in format_alerts
    text += "New alerts:\n" + "".join(["* %s\n" % format_alert(alert, node_map) for alert in new_alerts]) + "\n"
  File "/usr/local/lib/python3.6/site-packages/middlewared/alert/base.py", line 183, in <listcomp>
    text += "New alerts:\n" + "".join(["* %s\n" % format_alert(alert, node_map) for alert in new_alerts]) + "\n"
  File "/usr/local/lib/python3.6/site-packages/middlewared/alert/base.py", line 195, in format_alert
    return (f"{node_map[alert.node]} - " if node_map else None) + alert.formatted
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
 

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
No, rebooting didn't fix it. Well this is a whole different thread than where I started. Maybe I should start a new one.
 

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
Top