"Replication failed!" email notification appear once a minute

[Apollo]

I have a replication that is not able to finalize, and this is intended or partially intended as the PULL server is not available.
Anyway, what is really bothering me is the amount of emails sent. I would agree one email should be enough, but in this case one failed replication produced around 360 emails or so overnight. I can't wait to see what happens when all my replication task are failing.

Is there a way to change this behavior to only ackowneldge one instance of the failure?

 

[cyberjock]

There is no way to change the behavior. This is something that you might want to put a ticket in for. I'll be honest though, I'm not sure if this is something easily fixed in 9.3. I know there's some major reworking coming in 10.x, so I wouldn't be surprised if the ticket is accepted and then put in 10.x. But in any case, I'd definitely put the ticket in.

In my particular opinion (and I'm pretty conservative with communication to/from my server) I'd rather it send one email a minute while it's failing. The reason being is that:

1. If the PULL server were to come back online, the emails will stop, which will give you a time frame to go and see what was going on at that time.
2. Emails seem to get lost sometimes, and if it sends one email and then no more you might not see that one email. But if you get 150 emails, you'll definitely take notice. ;) (I'd rather take notice.. not to mention it's trivial to delete 100 emails or more).
3. I take the stance that emails of problems can never be sent too frequently. Emails cost nothing to send, cost nothing to process, and take no effort to delete. The real pain having to read them.

 

[Apollo]

Makes sense to a point

2. Emails seem to get lost sometimes, and if it sends one email and then no more you might not see that one email. But if you get 150 emails, you'll definitely take notice. ;) (I'd rather take notice.. not to mention it's trivial to delete 100 emails or more).
3. I take the stance that emails of problems can never be sent too frequently. Emails cost nothing to send, cost nothing to process, and take no effort to delete. The real pain having to read them.

True, but there is no way to scheme trough the details for each emails. All of them show the subject to be only "Replication failed". So several replication are not going to be easy to isolate.
At list if the subject would include the name of the snapshot then it would be easier to organize.
Also too many emails could potentially distract from more critical ones.
By the time I read through each email time has gone by.
I could create rules to filter emails but this is going to become difficult to manage.

 

[cyberjock]

But you don't have to read through them. Let's say you get 5000 emails (you've been a very bad boy and neglected your email for days).

You log in and realize that your server is pissed at you and emailing you every single minute for the last few days complaining about failing replication. After you've checked a few emails you'll see that they are all the same. So you sort by subject and delete them all. You could even sort by sender and delete them all if that works. Boom, done.

Even if you've got multiple replication tasks failing and you want to know about all of them, who cares? One you've got the replication fixed you'll either (1) look at the other tasks "just to make sure" all is well, only to find out they aren't, or (2) you'll continue to get emails that *another* replication task is failing.

This seems straight forward and dead simple to solve. Gotta delete en masse.

When I was in the Navy I had made an error in judgement and setup our virus server to point to my personal email if it had a problem. I figured it would be "just for the weekend" and it wouldn't matter. Well, Monday morning the email server is offline and I have 1.7 million emails. Delete en masse is how we got the server back online. I had real unread emails mixed in with the 1.7 million emails. Guess how I sorted that mess out? ;)

 

[trinxie]

I got the same "problem" this morning. I Upgraded my replication server yesterday and forgot to unlock the encrypted volume. This night I got about 600 emails for the "replication failed" error.
One suggestion is to reduce to one error per volume per night maybe? 600 is like 595 too many...
At least I know something was wrong which is good

 

[Apollo]

I got the same "problem" this morning. I Upgraded my replication server yesterday and forgot to unlock the encrypted volume. This night I got about 600 emails for the "replication failed" error.
One suggestion is to reduce to one error per volume per night maybe? 600 is like 595 too many...
At least I know something was wrong which is good :)

Indeed , and yet there is only one single unique alone e-mail produced for security run output warning, ie indicating a user didn't enter the correct password and that's only being prompted at 3:01 AM.

 

[spitfire]

Last night I've got more than 1000 e-mails overnight (yes, one THOUSAND) which is a real problem for me - as I was using my personal gmail account to both send and receive them. This morning I was not able to send any e-mails (exceeded daily sent e-mails quota), and my mail app on my phone didn't play nicely. Did someone open a ticket for this issue? If so, what's the number?

 

[trinxie]

I did not open a ticket. Have not done it before so I wanted to be sure how to do it. Must be easy to re-create.

 

[spitfire]

I did not open a ticket. Have not done it before so I wanted to be sure how to do it. Must be easy to re-create.

I did:
https://bugs.freenas.org/issues/8071

If anyone, has anything to add - please do.

 

[spitfire]

But you don't have to read through them. Let's say you get 5000 emails (you've been a very bad boy and neglected your email for days).

You log in and realize that your server is pissed at you and emailing you every single minute for the last few days complaining about failing replication. After you've checked a few emails you'll see that they are all the same. So you sort by subject and delete them all. You could even sort by sender and delete them all if that works. Boom, done.

Even if you've got multiple replication tasks failing and you want to know about all of them, who cares? One you've got the replication fixed you'll either (1) look at the other tasks "just to make sure" all is well, only to find out they aren't, or (2) you'll continue to get emails that *another* replication task is failing.

This seems straight forward and dead simple to solve. Gotta delete en masse.

When I was in the Navy I had made an error in judgement and setup our virus server to point to my personal email if it had a problem. I figured it would be "just for the weekend" and it wouldn't matter. Well, Monday morning the email server is offline and I have 1.7 million emails. Delete en masse is how we got the server back online. I had real unread emails mixed in with the 1.7 million emails. Guess how I sorted that mess out? ;)

Not that easy, if you've used a gmail account (like I did) or any other that has some arbitrary limit of e-mails that can be sent daily.
Not to mention getting an e-mail every minute for the same event does not make sense. It might make sense in the logs (most of the times it would not), but surely not for an e-mail notification..

 

[trinxie]

I agree with spitfire, it does not make sense. In previous releases we got one replication error per night if I remember right. Sounds reasonable for me.
What if I go for vacation and the power goes down on the replication side. Then I will receive these emails every night until you get back home.