SMTP - single session for multiple emails

More
3 months 4 weeks ago #190094 by bdeprez
Hi all,

Our SMTP infrastructure is going to implement new restrictions where a single sender host will only be allowed to open 30 sessions / minute with the SMTP server.

The current behavior of Limesurvey is as follows:
- open session
- send email 1
- close session
- open session
- send email 2
- close session

For us to adapt to the new restrictions, it would need to be:
- open session
- send email 1
- send email 2
- close session

Now, I have no idea if that is at all possible with Limesurvey?

Note: I can't use sendmail or the likes either. And I'm also still on 2.73.1 since IE is still very much the standard browser in our environment (Chrome is available but needs survey admins to install it first)

Thanks in advance,
B.

Please Iniciar sessão or Create an account to join the conversation.

LimeSurvey Partners
More
3 months 4 weeks ago #190113 by DenisChenu
I don't know if it's possible (out of LimeSurvey system …) see the rfc : tools.ietf.org/html/rfc821#section-3.1

But something i didn't understand

> Our SMTP infrastructure is going to implement new restrictions where a single sender host will only be allowed to open 30 sessions / minute with the SMTP server.

You need something, but your infrastructure can not offer ? Then change your infrastructure …

They can allow it for specific system , from specific user etc …

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
An error happen ? Before make a new topic : remind the Debug mode .

Please Iniciar sessão or Create an account to join the conversation.

More
3 months 4 weeks ago #190132 by jelo

DenisChenu wrote: I don't know if it's possible (out of LimeSurvey system …) see the rfc : tools.ietf.org/html/rfc821#section-3.1

What do you mean? I don't see any special there?

If LimeSurvey is running with a SMTP session restriction some issues will pop up.
E.g. when using notification mails. Depending on how the SMTP session restriction is done, respondents will not be able to submit a survey when notification emails are activated. The submission of an survey would trigger the smtp mail code and would hang there. I haven't tested it in 2019, so the code might have been improved. I don't use LimeSuvey for mailings (features are lacking and separating response data from contact data helps for data protection).

The meaning of the word "stable" for users
www.limesurvey.org/forum/development/117...ord-stable-for-users

Please Iniciar sessão or Create an account to join the conversation.

More
3 months 4 weeks ago - 3 months 4 weeks ago #190135 by DenisChenu

jelo wrote:

DenisChenu wrote: I don't know if it's possible (out of LimeSurvey system …) see the rfc : tools.ietf.org/html/rfc821#section-3.1

What do you mean? I don't see any special there?

OK, right phpMailer have such options

And more : i'm sure at 100% it was most easy to fix the restriction than to do a plugin …

See
1. gitlab.com/SondagesPro/mailing/eventBeforeTokenEmailExtended
2. SMTPKeepAlive

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
An error happen ? Before make a new topic : remind the Debug mode .
Last edit: 3 months 4 weeks ago by DenisChenu.

Please Iniciar sessão or Create an account to join the conversation.

More
3 months 4 weeks ago - 3 months 4 weeks ago #190136 by jelo

DenisChenu wrote: open session
- send email 1
- send email 2
- close session

I see. Yes, I don't think if LimeSurvey is used for e.g. mailing invitations, the SMTP is closed after every single emailadress. That makes no sense. Perhaps a glitch in the code, but not the intended behavior.

The default config had a 50 limit to prevent timeout in PHP.
$config['maxemails']          = 50; // The maximum number of emails to send in one go (this is to prevent your mail server or script from timeouting when sending mass mail)

@bdeprez might be able to explain when LimeSurvey is closing the session after every email (even if more than one email should be send).

The meaning of the word "stable" for users
www.limesurvey.org/forum/development/117...ord-stable-for-users
Last edit: 3 months 4 weeks ago by jelo.

Please Iniciar sessão or Create an account to join the conversation.

More
3 months 4 weeks ago #190137 by DenisChenu
Else :

See
1. gitlab.com/SondagesPro/mailing/eventBeforeTokenEmailExtended
2. SMTPKeepAlive

BUT : think it's mots easy to fix SMTP server …

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
An error happen ? Before make a new topic : remind the Debug mode .

Please Iniciar sessão or Create an account to join the conversation.

More
3 months 4 weeks ago #190157 by bdeprez
Hi all,

thanks for the quick reply!

@jelo - it does indeed close the connection after each email (see debug details):
Invitation sent to:1: Recipient 1 (recipient1@domain.com)
 
SMTP debug output:
2019-10-21 19:03:46	SERVER -> CLIENT: 220 mail.domain.com Microsoft ESMTP MAIL Service, Version: 7.0.6001.18000 ready at  Mon, 21 Oct 2019 20:03:46 +0100
2019-10-21 19:03:46	CLIENT -> SERVER: EHLO senderhost.domain.com
2019-10-21 19:03:46	SERVER -> CLIENT: 250-mail.domain.com Hello [10.198.34.175]
                   	                  250-TURN
                   	                  250-SIZE 2097152
                   	                  250-ETRN
                   	                  250-PIPELINING
                   	                  250-DSN
                   	                  250-ENHANCEDSTATUSCODES
                   	                  250-8bitmime
                   	                  250-BINARYMIME
                   	                  250-CHUNKING
                   	                  250-VRFY
                   	                  250 OK
2019-10-21 19:03:46	CLIENT -> SERVER: MAIL FROM:<survey@domain.com>
2019-10-21 19:03:46	SERVER -> CLIENT: 250 2.1.0 survey@domain.com....Sender OK
2019-10-21 19:03:46	CLIENT -> SERVER: RCPT TO:<srecipient1@domain.com>
2019-10-21 19:03:46	SERVER -> CLIENT: 250 2.1.5 recipient1@domain.com
2019-10-21 19:03:46	CLIENT -> SERVER: DATA
2019-10-21 19:03:46	SERVER -> CLIENT: 354 Start mail input; end with <CRLF>.<CRLF>
BODY REMOVED
2019-10-21 19:03:46	CLIENT -> SERVER: .
2019-10-21 19:03:46	SERVER -> CLIENT: 250 2.6.0  <80be62354285e767a248bb011be326e6@senderhost.domain.com> Queued mail for delivery
2019-10-21 19:03:46	CLIENT -> SERVER: QUIT
2019-10-21 19:03:46	SERVER -> CLIENT: 221 2.0.0 mail.domain.com Service closing transmission channel
 
Invitation sent to:	2: Recipient 2 (recipient2@domain.com)
 
SMTP debug output:
2019-10-21 19:03:46	SERVER -> CLIENT: 220 mail.domain.com Microsoft ESMTP MAIL Service, Version: 7.0.6001.18000 ready at  Mon, 21 Oct 2019 20:03:46 +0100
2019-10-21 19:03:46	CLIENT -> SERVER: EHLO senderhost.domain.com
2019-10-21 19:03:46	SERVER -> CLIENT: 250-mail.domain.com Hello [10.198.34.175]
                   	                  250-TURN
                   	                  250-SIZE 2097152
                   	                  250-ETRN
                   	                  250-PIPELINING
                   	                  250-DSN
                   	                  250-ENHANCEDSTATUSCODES
                   	                  250-8bitmime
                   	                  250-BINARYMIME
                   	                  250-CHUNKING
                   	                  250-VRFY
                   	                  250 OK
2019-10-21 19:03:46	CLIENT -> SERVER: MAIL FROM:<survey@domain.com>
2019-10-21 19:03:46	SERVER -> CLIENT: 250 2.1.0 survey@domain.com....Sender OK
2019-10-21 19:03:46	CLIENT -> SERVER: RCPT TO:<recipient2@domain.com>
2019-10-21 19:03:46	SERVER -> CLIENT: 250 2.1.5 recipient2@domain.com
2019-10-21 19:03:46	CLIENT -> SERVER: DATA
2019-10-21 19:03:46	SERVER -> CLIENT: 354 Start mail input; end with <CRLF>.<CRLF>
BODY REMOVED
2019-10-21 19:03:46	CLIENT -> SERVER: .
2019-10-21 19:03:46	SERVER -> CLIENT: 250 2.6.0  <4d0b80f6030962bcd78215a28fcf2d5a@senderhost.domain.com> Queued mail for delivery
2019-10-21 19:03:46	CLIENT -> SERVER: QUIT
2019-10-21 19:03:46	SERVER -> CLIENT: 221 2.0.0 mail.domain.com Service closing transmission channel
All emails were sent.

@DenisChenu:

We can't change out SMTP server settings - it will be a long road before that happens...

But thank you for your suggestions!

Would I need both 1. and 2. or would 2 just be sufficient?

I see in a topic on stackoverflow re item 2 (see: this link ):
Just set to true the SMTPKeepAlive property and after the bulk sending, call implicitly the SmtpClose() method.
 
$phpMailer = New PHPMailer();
$phpMailer->isSMTP();
$phpMailer->SMTPKeepAlive = true;
 
for ( ... ) {
    // Send your emails right away
    [ ... ]
}
 
$phpMailer->SmtpClose();
 

so I'll have to investigate where to add this "$phpMailer->SmtpClose();"

Thanks,
B.

Please Iniciar sessão or Create an account to join the conversation.

More
3 months 4 weeks ago - 3 months 4 weeks ago #190163 by DenisChenu

bdeprez wrote: We can't change out SMTP server settings - it will be a long road before that happens...

IT must be a services for other services … IT must not control other servives :|

bdeprez wrote: But thank you for your suggestions!

Would I need both 1. and 2. or would 2 just be sufficient?

No,

I think you need to take eventBeforeTokenEmailExtended and fix, and hack the way of set the phpMailer or create a plugin using this plugin.

Adding

$this->PhpMailer->SMTPKeepAlive = true;

inside setPhpMailer function

And

$this->PhpMailer->smtp->reset();

inside the reset part

And
Remlind to close smpt at end.

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
An error happen ? Before make a new topic : remind the Debug mode .
Last edit: 3 months 4 weeks ago by DenisChenu.

Please Iniciar sessão or Create an account to join the conversation.

More
3 months 1 week ago - 3 months 1 week ago #190932 by bdeprez
Took a while for me to get back to this but I think I'm halfway there... @DenisChenu

In your plugin, I added this:
$this->PhpMailer->SMTPKeepAlive = true;

and now it neatly keeps the session open and resets it after each email (that seems to be part of the standard behavior of phpmailer.php):
2019-11-09 21:18:23	CLIENT -> SERVER: RSET
2019-11-09 21:18:23	SERVER -> CLIENT: 250 2.0.0 Ok

But now I need to close the session all together at the end - and that's where I don't know how to proceed...

And
Remlind to close smpt at end.


I think the only place that would work is in the "tokens.php" but then how to invoke that action?

Suggestions?
Last edit: 3 months 1 week ago by bdeprez.

Please Iniciar sessão or Create an account to join the conversation.

More
3 months 1 week ago #190989 by DenisChenu

bdeprez wrote:
I think the only place that would work is in the "tokens.php" but then how to invoke that action?

Arg … right …

Totally unsure , since need to know the number still in queue ?

Maybe find the number with Yii::app()->getConfig('maxemails') and github.com/LimeSurvey/LimeSurvey/blob/ef...min/tokens.php#L1340 ?????

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
An error happen ? Before make a new topic : remind the Debug mode .

Please Iniciar sessão or Create an account to join the conversation.

More
2 weeks 1 day ago #193437 by Mazi
@bdeprez, would it be possible to share your solution later?

Best regards/Beste Grüße,
Dr. Marcel Minke
Need Help? We offer professional Limesurvey support
Contact: marcel.minke(at)survey-consulting.com
Want to use your survey offline -> www.offlinesurveys.com

Please Iniciar sessão or Create an account to join the conversation.

More
2 weeks 1 day ago #193442 by DenisChenu
Plugin :

1. get the mailer : github.com/LimeSurvey/LimeSurvey/blob/18...SenderToFrom.php#L39
2. Set it to keepalive $limeMailer->SMTPKeepAlive = true;

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
An error happen ? Before make a new topic : remind the Debug mode .

Please Iniciar sessão or Create an account to join the conversation.

More
2 weeks 1 day ago #193447 by Mazi
Thanks for the links, Denis. I am actually researching sample code for a feature to provide multiple SMTP servers at a single Limesurvey systems. Mainly to deliver 100.000 emails using different servers.
The idea is to have e.g. 5 different SMTP server settings and every 1000 emails the system uses a different setting for sending invitation emails.

Best regards/Beste Grüße,
Dr. Marcel Minke
Need Help? We offer professional Limesurvey support
Contact: marcel.minke(at)survey-consulting.com
Want to use your survey offline -> www.offlinesurveys.com

Please Iniciar sessão or Create an account to join the conversation.

More
2 weeks 1 day ago #193449 by DenisChenu
The new LimeMailer is done for this (in fact is done to do anything to email …)

You can set whole mailer config …

But maybe there are 3.X solution ?
Check gitlab.com/SondagesPro/mailing/smtpByUser

1. You can add a counter each time a mail is send
2. You can check this counter before sending any email
3. You can set the settings according to currrent counter

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
An error happen ? Before make a new topic : remind the Debug mode .

Please Iniciar sessão or Create an account to join the conversation.

More
2 weeks 1 day ago #193464 by jelo

DenisChenu wrote: The new LimeMailer is done for this

The new "LimeMailer" is what?

The meaning of the word "stable" for users
www.limesurvey.org/forum/development/117...ord-stable-for-users

Please Iniciar sessão or Create an account to join the conversation.

Começar agora!

Basta criar a sua conta e comece a usar o LimeSurvey hoje mesmo.

Registar agora
Subscreva o nosso boletim informativo!

Iniciar sessão